Interconnection network connecting operation-configurable nodes according to one or more levels of adjacency in multiple dimensions of communication in a multi-processor and a neural processor

ABSTRACT

A Wings array system for communicating between nodes using store and load instructions is described. Couplings between nodes are made according to a 1 to N adjacency of connections in each dimension of a G×H matrix of nodes, where G≧N and H≧N and N is a positive odd integer. Also, a 3D Wings neural network processor is described as a 3D G×H×K network of neurons, each neuron with an N×N×N array of synaptic weight values stored in coupled memory nodes, where G≧N, H≧N, K≧N, and N is determined from a 1 to N adjacency of connections used in the G×H×K network. Further, a hexagonal processor array is organized according to an INFORM coordinate system having axes at 60 degree spacing. Nodes communicate on row paths parallel to an FM dimension of communication, column paths parallel to an IO dimension of communication, and diagonal paths parallel to an NR dimension of communication.

RELATED U.S. APPLICATION DATA

The present application is a continuation-in-part of application Ser.No. 12/927,837 filed Nov. 27, 2010, which is a continuation ofapplication Ser. No. 12/477,232 filed Jun. 3, 2009, now U.S. Pat. No.7,886,128, which is a divisional of application Ser. No. 11/277,507filed Mar. 26, 2006, now U.S. Pat. No. 7,581,079, which claims thebenefit of U.S. Provisional Application No. 60/665,668 filed Mar. 28,2005 and U.S. Provisional Application No. 60/687,719 filed Jun. 6, 2005,all of which are incorporated by reference herein in their entirety.

FIELD OF INVENTION

The present invention relates to unique and improved methods andapparatuses for processor architecture and organizations of processorsand memory modules such that communication between the modules isefficient. More specifically, this invention concerns multiprocessorsystems having a shared memory interconnection network for communicationamong the processors and memory modules and an architecture andprocessor organization that efficiently supports such communication andneural processing.

BACKGROUND OF INVENTION

One of the problems associated with increasing performance inmultiprocessor parallel processing systems is the efficient accessing ofdata or instructions from memory. Having adequate memory bandwidth forsharing of data between processors is another problem associated withparallel processing systems. These problems are related to theorganization of the processors and memory modules and the processorarchitecture used for communication between a processor and memory andbetween processors. Various approaches to solving these problems havebeen attempted in the past, for example, array processors and sharedmemory processors.

Multiprocessor systems can be classified generally in terms of couplingstrength for communication between processors. Those multiprocessorsystems that communicate using a share memory facility between theprocessors and the shared memory over an interconnection network aregenerally considered tightly coupled. Loosely coupled multiprocessorsystems generally use an input/output (I/O) communication mechanism ineach processor, such as message passing, for communicating between theprocessors over an interconnection network. A wide variety ofinterconnection networks have been utilized in multiprocessing systems.For example, rings, bus connected, crossbar, tree, shuffle, omega,butterfly, mesh, hypercube, and ManArray networks, have been used inprior multiprocessor systems. From an application or user perspective,specific networks have been chosen primarily based upon performancecharacteristics and cost to implement tradeoffs.

A network for an application of a multiprocessor system is evaluatedbased on a number of characteristics. Parameters considered include, forexample, a network size of N nodes, where each node has L connectionlinks including input and output paths, a diameter D for the maximumshortest path between any two pair of nodes, and an indication of thecost C in terms of the number of connection paths in the network. A ringnetwork, for example, provides connections between adjacent processorsin a linear organization with L=2, D=N/2, and C=N. In another example, acrossbar switch network provides complete connectivity among the nodeswith L=N, D=1, and C=N². Table 1 illustrates these characteristics for anumber of networks where N is a power of 2.

Network of N nodes Links Diameter Cost N a power of 2 (L) (D) (C) Ring 2N/2 N BxB Torus for N = 2^(K) 4 B = 2^(K/2) 2N For K even & B = 2^(K/2)XD Hypercube for Log₂N Log₂N (X/2)N X = Log₂N XD ManArray hypercube 4 22^(2k−1)((4 + 3^(k−1)) − 1) for X = 2k and X even Crossbar N 1 N²

FIG. 1A illustrates a prior art 4×4 torus network 100 having sixteenprocessor (P) elements (PEs). Each PE supports four links in the regularnearest neighborhood connection pattern shown. The diameter is four,which is the maximum shortest path between any two nodes, such as, forexample, P00 and P22. The cost is thirty-two representing the thirty-twoconnections used to interconnect the PEs.

FIG. 1B illustrates a connectivity matrix 150 for the 4×4 torus network100 of FIG. 1A. Each of the sixteen PEs represents a column and a row ofthe matrix. A “1” in a cell of the connectivity matrix 150 indicatesthat the row PE connects to the column PE. For example, four “1”spopulate P21 row 154, indicating that P21 connects to P11, P20, P22, andP31. The connectivity matrix 150 is populated only with the nearestneighbor connections.

FIG. 2 illustrates a prior art 4×4 ManArray network 200, as illustratedin U.S. Pat. No. 6,167,502. The 4×4 ManArray network 200 has sixteenprocessors such as processor 1,3 (0110) 204. Each processor is connectedto a local cluster switch, such as local cluster switch 208 associatedwith a 2×2 processor cluster, such as, 2×2 processor cluster 212. In thecluster switch are a number of multiplexers which are connected to theprocessors to provide the interconnecting network for the sixteenprocessors. For example, each of the four processors in the 2×2processor cluster 212 connect to four multiplexers in the associatedlocal cluster switch 208. The 4×4 ManArray network 200 has an indicationof the cost C of 88 and a diameter of 2.

FIG. 3 illustrates a prior art shared memory processor 300 havingprocessor nodes P0-Pp-1 304, memory nodes Mo-Mm-1 306, input/output(I/O) nodes I/O0-I/Od-1 308 interconnected by a cross bar switch 310.The cross bar switch provides general data accessing between theprocessors, memory, and I/O. The processors typically interface tomemory over a memory hierarchy which typically locates instruction anddata caches local to the processors. The memories M0-Mm-1 typicallyrepresent higher levels of the memory hierarchy above the local caches.

The prior techniques of interconnecting memory and processors have tocontend with multiple levels of communication mechanisms and complexorganizations of control and networks.

SUMMARY OF THE INVENTION

It is appreciated that improvements to processor architecture, networkdesign, and organizations of processors and memory are desired. Suchimprovements are provided by multiple embodiments of the presentinvention. In one embodiment of the present invention a network isprovided. The network has groups of A_(g,h) nodes, each group having adifferent g that is the same for each A_(g,h) node in that group,gε{0,1, . . . , G−1} and for each group, hε{0,1, . . . , H−1}, and eachA_(g,h) node operable to output an A_(g,h) data value, wherein networknodes are identified according to a G×H matrix of nodes having a 1 to Nadjacency of connections between adjacent nodes in each dimension whichincludes wrap around adjacent nodes and G≧N and H≧N. The network alsohas groups of R_(g,h) nodes, each group having a different g that is thesame for each R_(g,h) node in that group, gε{0,1, . . . , G−1} and foreach group, hε{0,1, . . . , H−1}, each group of R_(g,h) nodes coupled toa corresponding group of A_(g,h) nodes according to a 1 to N adjacencyof connections in a first dimension, wherein each R_(g,h) node isoperable to select an A_(g,h) data value from a coupled A_(g,h) node andto output the selected A_(g,h) data value as an R_(g,h) data value. Thenetwork further has groups of S_(g,h) nodes, each group having adifferent g that is the same for each S_(g,h) node in that group,gε{0,1, . . . , G−1} and for each group, hε{0,1, . . . , H−1}, eachgroup of S_(g,h) nodes coupled to groups of R_(g,h) nodes according to a1 to N adjacency of connections in a second dimension, wherein eachS_(g,h) node is operable to select an R_(g,h) data value from a coupledR_(g,h) node and to output the selected R_(g,h) data value as an S_(g,h)data value.

In another embodiment of the present invention a network is provided.The network has a plurality of A nodes, each A node identified accordingto its position in a D-dimensional network and operable to output an Adata value, wherein the D-dimensional network is configured with nearestneighbor connectivity between adjacent nodes in each dimension ofcommunication. The network also has a plurality of D stages, one stagefor each dimension of communication of the D-dimensional network, each Anode coupled to a plurality of N first stage nodes according to anadjacency of nodes in a first dimension of communication, each of theplurality of N first stage nodes coupled to a plurality of N secondstage nodes according to an adjacency of nodes in a second dimension ofcommunication, and continuing until each of a plurality of N D−1 stagenodes are coupled to a plurality of N D stage nodes according to anadjacency of nodes in a D dimension of communication, wherein each nodeis configured to operate on a data value received from a coupled node ina previous stage that was initiated by the A data value output from eachof the A nodes.

In a further embodiment of the present invention a network comprising isprovided. The network has a plurality of M nodes, each M node identifiedaccording to its position in a D-dimensional network and operable tooutput an M data value, wherein the D-dimensional network is configuredwith nearest neighbor connectivity between adjacent nodes in eachdimension of communication. The network has a plurality of N first stagenodes coupled to each of the plurality of M nodes according to anadjacency of nodes in a first dimension of communication. The networkalso has a plurality of N second stage nodes coupled to each of theplurality of N first stage nodes according to an adjacency of nodes in asecond dimension of communication. The network stages continues up to aplurality of N D stage nodes coupled to each of a plurality of N D−1stage nodes to according to an adjacency of nodes in a D dimension ofcommunication, wherein each coupled first stage node is configured tooperate on an M data value from each coupled M node and to output afirst stage node result value, each coupled second stage node isconfigured to operate on a result value from each coupled first stagenode and to output a second stage node result value, and continuing upto each coupled D stage node is configured to operate on a result valuefrom each coupled D−1 stage node and to output a D stage node resultvalue.

These and other features, aspects, techniques and advantages of theinvention will be apparent to those skilled in the art from thefollowing detailed description, taken together with the accompanyingdrawings and claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A illustrates a prior art 4×4 torus network having sixteenprocessing elements (PEs);

FIG. 1B illustrates a connectivity matrix for the 4×4 torus network ofFIG. 1A;

FIG. 2 illustrates a prior art 4×4 ManArray network from U.S. Pat. No.6,167,502;

FIG. 3 illustrates a prior art shared memory processor;

FIG. 4A illustrates a Wings array memory (WAM) sixteen processor (16)network for store (S) operations in accordance with the presentinvention;

FIG. 4B illustrates the effective store connectivity of the WAM16Snetwork of FIG. 4A in accordance with the present invention;

FIG. 5A illustrates a WAM16 load (L) network for load operations inaccordance with the present invention;

FIG. 5B illustrates the effective load connectivity of the WAM16Lnetwork of FIG. 5A in accordance with the present invention;

FIG. 6A illustrates a connectivity matrix for store operations for theWAM16S network of FIG. 4A in accordance with the present invention;

FIG. 6B illustrates a connectivity matrix for load operations for theWAM16L network of FIG. 5A in accordance with the present invention;

FIG. 6C illustrates a connectivity matrix for communicating betweenprocessors by combining store WAM16S and load WAM16L operations inaccordance with the present invention;

FIG. 7 illustrates an alternative WAM16L network for the purpose ofshowing the symmetric nature of the WAM network in accordance with thepresent invention;

FIG. 8A illustrates a construction of a WAM network node using a four toone multiplexer with a fan out to three locations in accordance with thepresent invention;

FIG. 8B illustrates an alternative construction of a WAM network nodeusing three four to one multiplexers each with a single fan out to aseparate location in accordance with the present invention;

FIG. 9A illustrates a WAM sixty-four processor (64) store (WAM64S)network showing the scalable nature of the Wings array memory network inaccordance with the present invention;

FIG. 9B illustrates a general form of a store path selected from theWAM64S network of FIG. 9A in accordance with the present invention;

FIG. 9C illustrates a store path selected from the WAM64S network ofFIG. 9A in accordance with the present invention;

FIG. 9D illustrates a three dimensional organization of the twenty sevenmemory nodes and processor P_(2,2,2) of FIG. 9C in accordance with thepresent invention;

FIG. 9E illustrates a method of constructing a network in accordancewith the present invention;

FIG. 10A illustrates a generic type of prior art arithmetic instructionformat;

FIG. 10B illustrates a Wings basic arithmetic/logic instruction formatin accordance with the present invention;

FIG. 10C illustrates a Wings basic store instruction format inaccordance with the present invention;

FIG. 10D illustrates a Wings basic load instruction format in accordancewith the present invention;

FIG. 10E illustrates a Wings basic load immediate format in accordancewith the present invention;

FIG. 11A illustrates a Wings processor node for use with the WAMnetworks and using the Wings basic instruction formats in accordancewith an embodiment of the present invention;

FIG. 11B illustrates an example of a WAM processor system in accordancewith the present invention;

FIG. 11C illustrates a WAM16 processor subsystem with a set of processornodes, a WAM16S/WAM16L combined network, a first set of memories, and asecond set of memories in accordance with the present invention;

FIG. 11D illustrates a combined network node that combines a WAM16L nodeand a WAM16S node into a single node and illustrates the function aspectof the WAM nodes in accordance with the present invention;

FIG. 12A illustrates Wings processor node made up of an execution nodeand a memory node in accordance with an embodiment of the presentinvention;

FIG. 12B illustrates processor node made up of an execution node and amemory node in accordance with an embodiment of the present invention;

FIG. 13 illustrates a memory node to T node subsystem in accordance withthe present invention;

FIG. 14 illustrates an exemplary WAM16S network in a physical layoutform of the WAM 16 store (WAM16S) network of FIG. 4A in accordance withthe present invention;

FIG. 15 illustrates an exemplary WAM16L network physical layout form ofthe alternative WAM16L network of FIG. 7 in accordance with the presentinvention;

FIGS. 16A and 16B where FIG. 16A illustrates an exemplary combinednetwork node that combines a WAM load node and a WAM store node into asingle combined node where the load and store nodes support expandedfunction capabilities and where FIG. 16B illustrates another alternativeWAM network node constructed using three sub-node units comprising inputand output interfaces and node function units in accordance with thepresent invention;

FIG. 17 illustrates an exemplary layout of the WAM16S network of FIG. 4Acombined with the alternative WAM16L network of FIG. 7 in a physicallayout form in accordance with the present invention;

FIG. 18 illustrates a Wings array memory (WAM) twenty five processor(WAM25S) network for store (S) operations;

FIG. 19A illustrates a selected processor to memory path in a Wingsarray memory (WAM) forty nine processor (WAM49S) network for store (S)operations in accordance with the present invention;

FIG. 19B illustrates a general form of a double adjacency store pathselected from the WAM49S network of FIG. 19A in accordance with thepresent invention;

FIG. 19C illustrates an exemplary double adjacency store path selectedfrom the WAM49S network;

FIG. 20A illustrates a load path to a neuron processor Pgh in accordancewith the present invention;

FIG. 20B illustrates an exemplary memory T node system for theT_(g=2,h=2) node in accordance with the present invention;

FIG. 20C illustrates an exemplary memory T node system for the T_(2,1)node in accordance with the present invention;

FIG. 20D illustrates an exemplary memory T node system for the T_(2,3)node in accordance with the present invention;

FIG. 20E illustrates a node L22 which provides a summation of the T nodeoutputs generated in the previous stage in accordance with the presentinvention;

FIG. 21A illustrates a load path to a neuron processor Pghk inaccordance with the present invention;

FIG. 21B illustrates an exemplary Z_(ghk) node for use in a 3dimensional (3D) Wings neural network processor with each neuron havinga 5×5×5 array of synaptic weight values in accordance with the presentinvention;

FIG. 22 illustrates a P_(g,h,k) node in accordance with the presentinvention;

FIG. 23A illustrates a hexagonal processor array organized according toan INFORM coordinate system in accordance with the present invention;

FIG. 23B illustrates a Wings hexagonal array memory (WHAM) storeconfiguration of the hexagonal array of FIG. 23A based on a 1 to 3adjacency of connections in each dimension of communication with wraparound at the edge nodes of the hexagonal array in accordance with thepresent invention;

FIG. 24 illustrates an exemplary WHAM19S network layout of the hexagonalarray of FIG. 23A based on a 1 to 3 adjacency of connections in eachdimension of communication with wrap around at the edge nodes of thehexagonal array in accordance with the present invention;

FIG. 25A illustrates a first exemplary Wings packet formats inaccordance with the present invention;

FIG. 25B illustrates a second exemplary Wings packet formats inaccordance with the present invention; and

FIG. 26 illustrates an exemplary WAM processor in accordance with thepresent invention.

DETAILED DESCRIPTION

FIG. 4A illustrates a Wings array memory (WAM) sixteen processor (16)(WAM16S) network 400 for store (S) operations. A processor array 404 ofsixteen processors 405-420 are illustrated as nodes that each caninitiate a store operation to store data in a memory location in theWings array memory (WAM) 424 consisting of sixteen memory blocks425-440. The processor and memory block nodes are organized in lineararrays and identified according to a G×H matrix where, in this example,G equals four representing the number of rows in the matrix and H equalsfour representing the number of columns. A processor P_(g,h), a memoryblock M_(g,h), and internal nodes of the network are labeled in a row gby column h format where gε{0,1, . . . , G−1} and hε{0,1, . . . , H−1}.The processors are not directly connected to each other nor are thememory blocks directly connected to any of the other memory blocks.

A two stage WAM network 444 interconnects the processors 405-420 andmemory blocks 425-440 for store operations. A first stage of nodes aremultiplexers 445-460 which are labeled in a row g by column h R_(g,h)matrix. A second stage of nodes are multiplexers 465-480 which arelabeled in a row g by column h S_(g,h) matrix. The processors P_(g,h)each have an output, memory blocks M_(g,h) each have an input, andmultiplexers R_(g,h) and S_(g,h) each have three inputs and an output.The processors P_(g,h), the memory blocks M_(g),_(h), the multiplexersR_(g,h), and the multiplexers S_(g,h) are labeled in the figures as Pgh,Mgh, Rgh, and Sgh, respectively, for ease of notation and reference inthe figures. The first stage of multiplexers 445-460 are partitionedinto groups by rows of the G=4×H=4 matrix. For example, in the first rowg=0 of the processor matrix, the outputs of the processors 405-408 areconnected to the inputs of the multiplexers 445-448. For the next row,g=1, the outputs of the processors 409-412 are connected to the inputsof the multiplexers 449-452. The next row, g=2, the outputs of theprocessors 413-416 are connected to the inputs of the multiplexers453-456. The last row, g=3, processors 417-420 are connected tomultiplexers 457-460.

In each group, the connections are made according to an adjacency ofnodes in a first dimension, for example, P00 405 is connected to R00445, R01 446, and R03 448. P01 406 is connected to R00 445, R01 446, andR02 447. P02 407 is connected to R01 446, R02 447, and R03 448. P03 408is connected to R00 445, R02 447, and R03 448. Each processor in thesecond row group P10-P13 409-412, third row group P20-P23 413-416, andfourth row group P30-P33 417-420, are connected in a similar fashionaccording to their row adjacency to second row multiplexers R10-R13449-452, third row multiplexers R20-R23 453-456, and fourth rowmultiplexers R30-R33 457-460, respectively.

The first stage multiplexers 445-460 are connected to the second stagemultiplexers 465-480 according to an adjacency of nodes in a seconddimension, for example, the output of the multiplexer node R00 445 isconnected to the inputs of the multiplexer nodes S00 465, S10 469, andS30 477. In a similar fashion, R01 446 is connected to S01 466, S11 470,and S31 478. R02 447 is connected to S02 467, S12 471, and S32 479. R03448 is connected to S03 468, S13 472, and S33 480. The multiplexers inthe second row group R10-R13 449-452 are connected to the second stagemultiplexers according to their column adjacency, such that, R10 449 isconnected to S00 465, S10 469, and S20 473, R11 450 is connected to S01466, S11 470, and S21 474, R12 451 is connected to S02 467, S12 471, andS22 475, and R13 452 is connected to S03 468, S13 472, and S23 476. Thethird row group R20-R23 453-456 and the fourth row group R30-R33 457-460are connected in a similar fashion according to their column adjacencyassociated second stage multiplexers from the multiplexers 465-480.

Each output of the second stage multiplexers connects to the input oftheir associated memory block at the same row column position. Forexample, the output of the multiplexer S00 465 connects to the input ofthe memory block M00 425, the output of the multiplexer S01 466 connectsto the input of the memory block M01 426, and so forth. A processorexecuting a store operation can write data to a single memory block orcombinations of up to nine memory blocks from the memory array 424. Forexample, processor P21 can store data to memories in its connected groupof memory blocks M10 429, M20 433, M30 437, M11 430, M21 434, M31 438,M12 431, M22 435, and M32 439.

The adjacency of nodes is represented by a G×H matrix where the nodes ofthe matrix may be processors, memory blocks, multiplexers, or the like,generally, having nodes N_(g,h) where gε{0,1, . . . ,G-1} and hε{0,1, .. . ,H-1}. A connection network, such as the WAM16S network 400 of FIG.4A, may be generalized as having a first set of nodes, such as processornodes P_(g,h), for example, connects to a second set of nodes R_(g,h)which connects to a third set of nodes S_(g,h). The third set of nodesS_(g,h) then connects to a fourth set of nodes, such as memory blocknodes M_(g,h), for example. The store connectivity of the nodes can beviewed as having nodes R_(g,h) connect as follows:

Inputs Connects to the out- of Node puts of the Nodes Where R_(g, h)P_(g, h), P_(g, h+1), and h + 1 wraps to 0 when h + 1 = H and P_(g, h−1)h − 1 wraps to H − 1 when h − 1 = −1

The nodes S_(g,h) connect as follows:

Inputs Connects to the out- of Node puts of the Nodes Where S_(g, h)R_(g, h), R_(g+1, h), and g + 1 wraps to 0 when g + 1 = G and R_(g−1, h)g − 1 wraps to G − 1 when g − 1 = −1

The nodes M_(g,h) connect as follows:

Input of Node Connects to the output of the Node M_(g, h) S_(g, h)

For the example WAM16S network 400 of FIG. 4A, the nodes R_(g,h) connectas follows:

Inputs Connects to the out- of Node puts of the Nodes Where R_(g, h)P_(g, h), P_(g, h+1), and h + 1 wraps to 0 when h + 1 = 4 and P_(g, h−1)h − 1 wraps to 4 − 1 = 3 when h − 1 = −1The nodes S_(g,h) connect as follows:

Inputs Connects to the out- of Node puts of the Nodes Where S_(g, h)R_(g, h), R_(g+1, h), and g + 1 wraps to 0 when g + 1 = 4 and R_(g−1, h)g − 1 wraps to 4 − 1 = 3 when g − 1 = −1The nodes M_(g,h) connect as follows:

Input of Node Connects to the output of the Node M_(g, h) S_(g, h)

The store connectivity of the nodes can also be viewed as having nodesP_(g,h) connect as follows:

Output Connects to an of Node input of the Nodes Where P_(g, h)R_(g, h), R_(g, h+1), and h + 1 wraps to 0 when h + 1 = H and R_(g, h−1)h − 1 wraps to H − 1 when h − 1 = −1The nodes R_(g,h) connect as follows:

Output Connects to an of Node input of the Nodes Where R_(g, h)S_(g, h), S_(g+1, h), and g + 1 wraps to 0 when g + 1 = G and S_(g−1, h)g − 1 wraps to G − 1 when g − 1 = −1The nodes S_(g,h) connect as follows:

Output of Node Connects to the input of the Node S_(g, h) M_(g, h)

This store connectivity is more clearly shown in FIG. 4B whichillustrates the effective store connectivity 485 of the WAM16S network400 of FIG. 4A. FIG. 4B is an overhead view of the memory array 424 ofFIG. 4A (octagonal blocks) overlaid upon the processor array 404 of FIG.4A (square blocks). The effective store paths between processors andmemories are obtained through the use of the two stage WAM network 444of FIG. 4A. Such effective store paths are shown as arrow linesconnecting a processor to a memory block. A store path between processorP_(g,h) and memory M_(g,h), such as between P21 414 and M21 434, isshown as a short arrow line beginning from the processor label P_(g,h)and pointing to the memory M_(g,h) block. Each memory block can bereached for storing data from a neighborhood of nine processors.

FIG. 5A illustrates a Wings array memory (WAM) sixteen processor (16)(WAM16L) network 500 for load (L) operations. A processor array 504 ofsixteen processors 505-520 are illustrated as nodes that each caninitiate a load operation to fetch data from a memory location in theWings array memory (WAM) 524 consisting of sixteen memory blocks525-540. The processor and memory block nodes are organized in a lineararray and identified according to a G×H matrix where G equals fourrepresenting the number of rows in the matrix and H equals fourrepresenting the number of columns. A processor P_(g,h) and a memoryblock M_(g,h) are labeled in a row g by column h format where gε{0,1, .. . , G−1} and hε{0,1, . . . , H−1}. The processors are not directlyconnected to each other nor are the memory blocks directly connected toany of the other memory blocks.

A two stage WAM network 544 interconnects the processors 505-520 andmemory blocks 525-540 for load operations. A first stage of nodes aremultiplexers 545-560 which are labeled in a row column T_(g,h) matrix. Asecond stage of nodes are multiplexers 565-580 which are labeled in arow column L_(g,h) matrix. The processors P_(g,h) each have an input,memory blocks M_(g,h) each have an output, and multiplexers T_(g,h) andL_(g,h) each have three inputs and an output. The processors P_(g,h),the memory blocks M_(g,h), the multiplexers T_(g,h), and themultiplexers L_(g,h) are labeled in the figures as Pgh, Mgh, Tgh, andLgh, respectively, for ease of notation and reference in the figures.The first stage of multiplexers 545-560 are partitioned into groups byrows of the G=4×H=4 matrix. For example, in the first row g=0 of thememory matrix, memories 525-528 are connected to multiplexers 545-548.For the next row, g=1, memories 529-532 are connected to multiplexers549-552. The next row, g=2, memories 533-536 are connected tomultiplexers 553-556. The last row, g=3, memories 537-540 are connectedto multiplexers 557-560.

In each group, the connections are made according to an adjacency ofnodes in a first dimension, for example, M00 525 is connected to T00545, T01 546, and T03 548. M01 526 is connected to T00 545, T01 546, andT02 547. M02 527 is connected to T01 546, T02 547, and T03 548. M03 528is connected to T00 545, T02 547, and T03 548. Each memory block in thesecond row group M10-M13 529-532, third row group M20-M23 533-536, andfourth row group M30-M33 537-540, are connected in a similar fashionaccording to their row adjacency to second row multiplexers T10-T13549-552, third row multiplexers T20-T23 553-556, and fourth rowmultiplexers T30-T33 557-560, respectively.

The first stage multiplexers 545-560 are connected to the second stagemultiplexers 565-580 according to an adjacency of nodes in a seconddimension, for example, T00 545 is connected to L00 565, L10 569, andL30 577. T01 546 is connected to L01 566, L11 570, and L31 578. T02 547is connected to L02 567, L12 571, and L32 579. T03 548 is connected toL03 568, L13 572, and L33 580. The multiplexers in the second row groupT10-T13 549-552 are connected to the second stage multiplexers accordingto their column adjacency, such that, T10 549 is connected to L00 565,L10 569, and L20 573, T11 550 is connected to L01 566, L11 570, and L21574, T12 551 is connected to L02 567, L12 571, and L22 575, and T13 552is connected to L03 568, L13 572, and L23 576. The third row groupT20-T23 553-556 and the fourth row group T30-T33 557-560 are connectedin a similar fashion according to their column adjacency associatedsecond stage multiplexers.

Each output of the second stage multiplexers connects to the load inputof their associated processors at the same row column position. Forexample, the output of the multiplexer L00 565 connects to the input ofprocessor P00 505, the output of the multiplexer L01 566 connects to theinput of processor P01 506, and so forth. A processor executing a loadoperation can select a memory block from a group of nine memory blocksto fetch data from the selected memory block. For example, processor P21514 can load data from memories in its connected group of memory blocksM10 529, M20 533, M30 537, M11 530, M21 534, M31 538, M12 531, M22 535,and M32 539. Load addresses may follow connection paths in a networkconfiguration such as the WAM16S network 400 of FIG. 4A, for example toprovide memory addresses to selected memories as part of a loadoperation. Alternative methods to handle address paths is discussed inmore detail below.

The adjacency of nodes is represented by a G×H matrix where the nodes ofthe matrix may be processors, memory blocks, multiplexers, or the like,generally, having nodes N_(g,h) where gε{0,1, . . . ,G-1} and hε{0,1, .. . ,H-1}. A connection network, such as the WAM16L network 500 of FIG.5A, may be generalized as having a first set of nodes, such as memorynodes M_(g,h), for example, connects to a second set of nodes T_(g,h)which connects to a third set of nodes L_(g,h). The third set of nodesL_(g,h) then connects to a fourth set of nodes, such as processor nodesP_(g,h), for example. The load connectivity of the nodes can be viewedas having nodes T_(g,h) connect as follows:

Inputs Connects to the out- of Node puts of the Nodes Where T_(g, h)M_(g, h), M_(g, h+1), and h + 1 wraps to 0 when h + 1 = H and M_(g, h−1)h − 1 wraps to H − 1 when h − 1 = −1

The nodes L_(g,h) connect as follows:

Inputs Connects to the out- of Node puts of the Nodes Where L_(g, h)T_(g, h), T_(g+1, h), and g + 1 wraps to 0 when g + 1 = G and T_(g−1, h)g − 1 wraps to G − 1 when g − 1 = −1

The nodes P_(g,h) connect as follows:

Input of Node Connects to the output of the Node P_(g, h) L_(g, h)

For the example WAM16L network 500 of FIG. 5A, the nodes T_(g,h) connectas follows:

Inputs Connects to the out- of Node puts of the Nodes Where T_(g, h)M_(g, h), M_(g, h+1), and h + 1 wraps to 0 when h + 1 = 4 and M_(g, h−1)h − 1 wraps to 4 − 1 = 3 when h − 1 = −1The nodes L_(g,h) connect as follows:

Inputs Connects to the out- of Node puts of the Nodes Where L_(g, h)T_(g, h), T_(g+1, h), and g + 1 wraps to 0 when g + 1 = 4 and T_(g−1, h)g − 1 wraps to 4 − 1 = 3 when g − 1 = −1The nodes P_(g,h) connect as follows:

Input of Node Connects to the output of the Node P_(g, h) L_(g, h)

This load connectivity is more clearly shown in FIG. 5B whichillustrates the effective load connectivity 585 of the WAM16S network500 of FIG. 5A. FIG. 5B is an overhead view of the processor array 504of FIG. 5A (square blocks) overlaid upon the memory array 524 of FIG. 5A(octagonal blocks). The effective load paths between memories andprocessors are obtained through the use of the two stage WAM network 544of FIG. 5A. Such effective load paths are shown as arrow linesconnecting a memory block to a processor. A load path between memoryM_(g,h) and processor P_(g,h), such as between M21 534 and P21 514, isshown as a short arrow line beginning from the memory M_(g,h) block andpointing to the processor P_(g,h). Each processor can be reached byloading data from a neighborhood of nine memory blocks.

FIG. 6A illustrates a store connectivity matrix 600 for store operationsfor the WAM16S network 400 of FIG. 4A. The processors are organized inthe same linear order as the processor array 404 shown in the WAM16Snetwork 400. The memories are organized in the same linear order as theWings array memory (WAM) 424 shown in the WAM16S network 400. Inaddition to the processor and memory labels used in the WAM16S network400, the processors and memories have a Gray encoded label underneaththe P_(g,h) and M_(g,h) labels. A 1 in a cell of the store connectivitymatrix 600 indicates that a processor on the same row as the cell has astore connection to a memory block on the same column as the cell. Forexample, the connectivity of the processors in processor group 602having processors P10, P11, P12, and P13 connecting to memory blocks inthe three memory block groups 604, 606, and 608 is indicated by “1s” asconnection points in circled connection sub-matrices 610, 612, and 614.

FIG. 6B illustrates a load connectivity matrix 630 for load operationsfor the WAM16L network 500 of FIG. 5A. The processors are organized inthe same order as the processor array 504 in the WAM16L network 500. Thememories are organized in the same linear order as the Wings arraymemory (WAM) 524 shown in the WAM16L network 500. In addition to theprocessor and memory labels used in the WAM16L network 500, theprocessors and memories have a Gray encoded label underneath the P_(g,h)and M_(g,h) labels. A 1 in a cell indicates that a memory block on thesame row as the cell has a load connection to a processor on the samecolumn as the cell.

FIG. 6C illustrates a connectivity matrix 670 for communicating betweenprocessors by combining store operations on the WAM16S network 400 andload operations on the WAM16L network 500. The connectivity matrix 670is obtained by multiplying the store connectivity matrix 600 with theload connectivity matrix 630. Such multiplication produces thecompletely connected matrix 670 shown in FIG. 6C. The advantageindicated by the completely connected matrix 670 is that completeconnectivity is achieved with less connection cost than a cross barswitch. It is also possible to pipeline stores and loads such that aneffective shortened cycle communication throughput may be obtained whilestill achieving complete connectivity. For example, with store and loadexecution times of a single cycle, an effective single cyclecommunication throughput may be obtained by overlapping store and loadoperations using software pipelining methods.

FIG. 7 illustrates an alternative WAM16L network 700 for the purpose ofshowing the symmetric nature of the WAM network. Both the WAM16L network500 and the WAM16L network 700 have the same load connectivity matrixand can be used interchangeably.

The adjacency of nodes is represented by a G×H matrix where the nodes ofthe matrix may be processors, memory blocks, multiplexers, or the likehaving nodes N_(g,h) where gε{0,1, . . . ,G-1} and hε{0,1, . . . ,H-1}.A connection network, such as the alternative WAM16L network 700 of FIG.7, may be generalized as having a first set of nodes, such as memorynodes M_(g,h), for example, connects to a second set of nodes T_(g,h)which connects to a third set of nodes L_(g,h). The third set of nodesL_(g,h) then connects to a fourth set of nodes, such as processor nodesP_(g,h), for example. The load connectivity of the nodes can be viewedas having nodes T_(g,h) connect as follows:

Inputs Connects to the out- of Node puts of the Nodes Where T_(g, h)M_(g, h), M_(g+1, h), and g + 1 wraps to 0 when g + 1 = G and M_(g−1, h)g − 1 wraps to G − 1 when g − 1 = −1The nodes L_(g,h) connect as follows:

Inputs Connects to the out- of Node puts of the Nodes Where L_(g, h)T_(g, h), T_(g, h+1), and h + 1 wraps to 0 when h + 1 = H and T_(g, h−1)h − 1 wraps to H − 1 when h − 1 = −1The nodes P_(g,h) connect as follows:

Input of Node Connects to the output of the Node P_(g, h) L_(g, h)

FIG. 8A illustrates a WAM network node 800 constructed using a three toone multiplexer 802 with a fan out to three locations 804-806. Themultiplexer has three inputs 809-811 as selected by mpx_(gh)(0:1)control signals 812. The states of the control signals 812 are shown incolumnar format 814 inside the multiplexer 802. When the control signals812 are in a specific state, the input associated with that state istransferred to the multiplexer output that fans out to three places804-806. For example, multiplexer control signals 812 set at “10” causethe value on input 810 to be sent to the three fan out locations804-806. The WAM network node 800 would be suitable for using as nodesin the WAM16S Rxx nodes 445-460 of FIG. 4A, Sxx nodes 465-480 of FIG.4A, WAM16L Txx nodes 545-560 of FIG. 5A, Lxx nodes 565-580 of FIG. 5A,alternative WAM16L Txx nodes 745-760 of FIG. 7, and Lxx nodes 765-780 ofFIG. 7.

FIG. 8B illustrates an alternative WAM network node 850 constructedusing three three to one multiplexers 852-854 each with a single fan out856-858 to a separate location. The external inputs 859-861 to thealternative WAM network node 850 have the same source as the inputsignals 809-811 of the WAM network node 800 of FIG. 8A. Each output856-858 of the alternative WAM network node 850 is separately sourced byits associated multiplexer 852-854, respectively. Since there are three3 to 1 multiplexers 852-854 in the alternative WAM network node 850,there are three sets of control signals with two lines each comprisingmpxgh(0:5) 864 required to appropriately control the three multiplexers852-854.

FIG. 9A illustrates a WAM sixty-four processor (64) store (WAM64S)network 900 showing the scalable nature of the Wings array memorynetwork. Each group of 16 processors 902, 904, 906, and 908 areconnected to a WAM16S network 910, 912, 914, and 916, respectively. TheWAM16S networks 910, 912, 914, and 916 are of the same type as theWAM16S network 400. Note that the processor notation, the multiplexernode notation, and the memory notation are based on G×H×K 3 dimensional(3D) cube organization, where G represents the number of rows on aplane, H represents the number of columns on the plane, and K representsthe number of planes in the 3D cube organization. A processor P_(g,h,k),a memory M_(g,h,k), a node R_(g,h,k), a node S_(g,h,k), and a nodeV_(g,h,k) are labeled in a row g by column h by plane k format wheregε{0,1, . . . ,G-1}, hε{0,1, . . . ,H-1}, and kε{0,1, . . . ,K-1}. TheWAM 64S network has G=4, H=4, and K=4. The processors P_(g,h,k), thememory blocks M_(g,h,k), the multiplexers R_(g),_(h),_(k), themultiplexers S_(g,h,k), and the multiplexers V_(g,h,k) are labeled inthe figures as Pghk, Mghk, Rghk, Sghk, and Vghk, respectively, for easeof notation and reference in the figures. The WAM64S network has threestages, two stages for the four WAM16S networks 910, 912, 914, and 916and one stage 918 for the K planes that connects to the 64 memory blocks920, 922, 924, and 926. A WAM64L network would be symmetric to theWAM64S network 900 in the same manner that the WAM16L network 700 issymmetric to the WAM16S network 400.

FIG. 9B illustrates a general form of a store path 930 selected from theWAM64S network 900. The store path begins at P_(g,h,k) 932 connecting toa first stage 933 of a WAM16S network to three R nodes 934-936. Thethree R nodes 934-936 connect to a second stage 937 of the WAM16Snetwork to nine S nodes 938-946. The nine S nodes 938-946 connectthrough a WAM network stage 947 to twenty seven V nodes 948 that eachconnect directly to a corresponding memory block in the twenty sevenmemory blocks 949.

FIG. 9C illustrates a store path 950 selected from the WAM64S network900. The store path 950 begins at P_(2,2,2) 952. This store path 950 isformed by substituting g=2, h=2, and k=2 in the subscripted notation ofthe general form of a store path 930 in FIG. 9B. The node numbers wrapwithin the range 0-3 for rows g, columns h, and planes k. An examplememory node is memory node M_(3,2,1) 954.

FIG. 9D illustrates a three dimensional organization 960 of the twentyseven memory nodes and processor P_(2,2,2) 952 of FIG. 9C. The storepath begins at P_(2,2,2) 952 and connects to the twenty seven memorynodes, such as memory node M_(3,2,1) 954.

FIG. 9E illustrates a method 970 of constructing a network in accordancewith the present invention. The method starts in step 971 where anetwork of nodes is identified by gε{0,1, . . . , G−1}, hε{0,1, . . . ,H−1}, kε{0,1, . . . , K−1}, zε{0,1, . . . , Z−1} and iε{0,1, . . . , D}where D is the number of dimensions. In step 972, variables i, g, h, k,. . . , z are initialized to zero.

For i=0 step 974, the first stage of the network is constructedconnecting node N(i)_(g,h,k, . . . , z) to nodeN(i+1)_(g,h,k, . . . , z) and to node N(i+1)_(g,h+1,k, . . . , a) and toN(i+1)_(g,h−1,k, . . . , z) where h+1 wraps to 0 when h+1=H and h−1wraps to H−1 when h−1=−1. In step 978, the variable h is incrementedby 1. In step 979 it is determined whether h=H. If h does not equal H,then the method returns to step 974. If h does equal H, then the methodproceeds to step 980.

In step 980, the variable h is set to zero and the variable g isincremented by 1. In step 981, it is determined whether g=G. If g doesnot equal 0, then the method returns to step 974. If g does equal G,then the method proceeds to step 982.

In step 982, the variable g is set to zero and the variable k isincremented by 1. The method 970 continues in like manner for thedimensions up to the test for the last dimension in step 983. In step983, it is determined whether z=Z. If z does not equal Z, then themethod returns to step 974. If z does equal Z, then the method proceedsto step 984.

In step 984, the variable z is set to zero and the variable i isincremented by 1. In step 985, it is determined whether i=D. If i doesnot equal D, then the method proceeds to step 975 with i=1. If i doesequal D, then the method stops at step 986 having constructed thenetwork.

For i=1 step 975, the second stage of the network is constructedconnecting node N(i)_(g,h,k . . . , z) to node N(i+1)_(g,h,k, . . . , z)and to node N(i+1)_(g+1,h,k, . . . , z) and toN(i+1)_(g−1,h,k, . . . , z) where g+1 wraps to 0 when g+1=G and g−1wraps to G−1 when g−1=−1. In step 978, the variable h is incrementedby 1. From step 975, the method proceeds to step 978 and the process isrepeated from step 978 through to the step 984. In step 984, thevariable z is set to zero and the variable i is incremented by 1. Themethod continues constructing stages of the network until the point isreached where i=D−1. In step 985 at this point, the process proceeds tostep 976 to construct the last stage of the network. Once the last stageof the network has been constructed, the method returns to step 984 andincrements the variable i by 1, such that i=D. In step 985, it isdetermined that i=D and the method proceeds to step 986 havingconstructed the network. It is noted that steps 988 are adjusted for thenumber of dimensions D of the network to be constructed. For example, ifD=2, as would be the case for the WAM16S network 400 of FIG. 4A, thenonly variables g and h would be required and steps 982 through steps 983would not be required. Also, step 984 would be adjusted to g=0, i=i+1.

The WAM16S network 400 of FIG. 4A may be constructed by use of themethod 970 where the dimensions (D) is 2. The method 970 would for D=2follow the steps illustrated in FIG. 9E including step 974 and step 975.Step 974 for i=0 and steps 988 adjusted for D=2 are used to constructthe first stage of the WAM16S network 400 between the processors P00 405through P33 420 and the multiplexers R00 445 through R33 460. Step 975for i=1 and steps 988 adjusted for D=2 are used to construct the secondstage of the WAM16S network 400 between the multiplexers R00 445 throughR33 460 and the multiplexers S00 465 through S33 480.

FIG. 10A illustrates a generic type of arithmetic instruction format1000. The arithmetic instruction 1000 is made up of a 6-bit opcode 1002,a 5-bit Rt register target field 1004, a 5-bit Rx register source field1006, a 5-bit Ry register source field 1008, and an 11-bit instructionspecific field 1010. This format is typical for a processor having acentral register file from which arithmetic operands are sourced andarithmetic results are targeted. A thirty two entry register file of,for example, 32-bits, organized as a 32×32-bit multi-port register file,is a typical processor register file requiring 5-bit addressing for eachport for direct access of operands. In a memory to memory processorwhich accesses operands from a memory, the specification of the sourceand target addresses in the arithmetic instruction typicallyaccommodates a wider addressing range. The wider addressing range isobtained either directly through wider operand address fields in aninstruction or through indirect forms of addressing using externaladdressing registers set up ahead of time.

In most processors, a fixed instruction format size is used, such as, 8,16, 24, 32 and 64 bits or a combination of such instruction formats.FIG. 10A shows one such 32-bit instruction format 1000. The spaceallocated in the 32-bit instruction format 1000 for three operandaddress fields 1004, 1006, and 1008 is necessarily limited, since theother instruction bits, such as 1002 and 1010, are required to specifyoperations necessary in order to execute the instruction as specified bythe processor's architecture. In order to break this limitation andprovide greater flexibility in specifying operand addresses, forexample, with greater range and flexible accessing methods, a newprocessor architecture, referenced as the Wings architecture, splits atypical instruction format into three separate new types of instructionformats each more optimally organized for their intended purposes. Afirst instruction format, an arithmetic/logic instruction format 1020,is shown in FIG. 10B to be used to specify arithmetic, logical, shift,bit manipulation, and the like operations. A second instruction format,a store instruction format 1040, is shown in FIG. 10C to be used tospecify operations to store results of arithmetic operations to memory.A third instruction format, a load instruction format 1060, is shown inFIG. 10D to be used to specify the accessing of operand data from memoryfor delivery to execution units. These and other variations arediscussed further below.

For example, FIG. 10B illustrates a Wings basic arithmetic/logic (AL)instruction format 1020 having 12-bits to define the operation. The ALformat 1020 has no memory source or target operand address fields. A6-bit opcode field 1022, a 3-bit data type (Dtype) 1024, and threearithmetic/logic instruction specific bits 1026 are all that is requiredto specify an arithmetic operation in the 12-bit AL instruction format1020. The Wings processor architecture specifies that whatever data isat the inputs to an AL unit at the start of an execution cycle that isthe data received in the AL unit and operated on by the AL unit. TheWings processor architecture also specifies that the results ofexecution are available at the output of the AL unit at the end of theexecution cycle or cycles. An AL instruction does not specify a targetstorage address in a central register file or a memory unit where theresults may be stored. In order to provide operands to an AL unit andstore results from an AL unit, an AL instruction must be paired with aload and a store instruction or other such instruction or instructionsto provide source operands and to take result operands for furtherprocessing or storage.

For example, FIG. 10C illustrates a Wings basic store instruction format1040 having 19-bits to define the operation. The store instructionformat 1040 uses a 3-bit store opcode 1042, two store instructionspecific bits 1044, a 4-bit direction/memory bank (MemBank) selectionfield 1046, and a 10-bit memory address 1048 in the 19-bit instructionformat. As specified by the opcode 1042 or in combination with the storeinstruction specific bits 1044, the store instruction causes a result tobe taken from a specified execution unit and store the result to thetarget memory address. The target memory address is determined from thecombination of the 4-bit direction/MemBank selection field 1046 and the10-bit memory address 1048. Direct, indirect, and other addressing formsmay be specified using separate addressing registers if required.

FIG. 10D illustrates a Wings basic load instruction format 1060 having19-bits to define the operation. The load instruction format uses a3-bit load opcode 1062, two load instruction specific bits 1064, a 4-bitdirection/memory bank (MemBank) selection field 1066, and a 10-bitmemory address 1068 in the 19-bit instruction format. As specified bythe opcode 1062 or in combination with the load instruction specificbits 1064, the load instruction fetches at least one source operand froma specified memory address for delivery to an execution unit. The memoryaddress is determined from the combination of the 4-bitdirection/MemBank selection field 1066 and the 10-bit memory address1068. Direct, indirect, and other addressing forms may be specifiedusing separate addressing registers if required. FIG. 10E illustrates aWings basic load immediate format 1080 having 19-bits to define theoperation. The load immediate format uses a 3-bit load immediate opcode1082 and a 16-bit immediate field 1088 in the 19-bit instruction format.The 3-bit load immediate opcode 1082, for example, may specify theexecution unit that is to use the immediate data.

It is anticipated the depending upon the application the processorarchitecture may expand or contract the illustrated instruction formats.For example, 8-bit arithmetic and 16-bit load and store instructionformats, and 16-bit arithmetic and 24-bit load and store instructionformats can be envisioned, as well as other variations, such as, 14-bitarithmetic and 25-bit load and store instruction formats. Theinstruction format is determined primarily from the number of and typeof operations to be specified for each class of instruction.

A secondary consideration may be how the instructions are packed forstorage as programs in external memory. For example, with use of baseaddress registers local in the PEs, a dual load instruction may bespecified that selects two source operands from blocks of memory bygenerating two addresses. The dual load instruction would be used inplace of two single load instructions. With a dual load instructionformat of 27-bits, a store instruction of 23-bits, and an arithmeticinstruction of 14-bits, a packed instruction storage space of 64-bitswould be required. The packed instruction storage space could beunpacked locally to the processor when loading instruction memories, forexample, as may be specified in direct memory access (DMA) typeoperations. Instruction memories, such as the execution unit instructionmemories of a Wings processor may be used. See U.S. ProvisionalApplication Ser. No. 10/648,154 entitled “Methods and Apparatus ForMeta-Architecture Defined Programmable Instruction Fetch FunctionsSupporting Assembled Variable Length Instruction Processors”, which isincorporated by reference in its entirety.

FIG. 11A illustrates a Wings processor node 1100 for use with the WAMnetworks, such as the WAM16S network 400, WAM 16L network 500 and 700,and WAM64S network 900. The Wings processor node 1100 uses the Wingsbasic instruction formats, 1020, 1040, 1060, and 1080. The Wingsprocessor node 1100 consists of a processor P_(g,h) 1104 with inputconnections for instruction memory addresses WinF-IM0 address andcontrols 1105, WinF-IM1 address and controls 1106, and WinF-IM2 addressand controls 1107. The processor P_(g,h) 1104 has output connections forWAM network connections 1109-1114 which are described in more detailbelow.

As noted above, the 12-bit arithmetic and 19-bit load and storeinstruction formats are one set of example formats that can be specifiedfor the processor nodes. Depending upon the application, the number andtype of unique instructions may require different instruction formats inorder to meet the requirements. It was also noted that it is desirableto optimize the instruction format to the needs of the instruction type,such as arithmetic/logic instructions, load and store instructions forexample. Since the instruction formats may take various numbers of bits,an architecture supporting a wide variety of formats is required. TheWings architecture, as described in US Patent Application Publication US2004/0039896, is an architecture that would allow different instructionsizes for each instruction type supported by a separate instructionmemory unit. The Wings architecture supplies instruction addresses tolocal instruction memories in each processor, such as load instructionmemory IM0 1116, arithmetic instruction memory IM1 1117, and storeinstruction memory IM2 1118 to select an instruction from each memory.The selected instruction is supplied on individual instruction buses toseparate decode units 1120-1122 and then executed in each separateexecution unit 1124-1126, respectively.

The load execute unit 1124 generates a data fetch address or loadaddress for each load instruction supplied by the load instructionmemory IM0 1116. For example, if two load instructions were suppliedthen two load addresses and network opcodes would be generated, such asload address 1 & load network 1 opcode 1109 and load address 2 & loadnetwork 2 opcode 1110. These fetch addresses and network opcodes are setthrough the network to each multiplexer node that is under control ofthe processor. In the WAM16L network 700, each processor node P_(g,h),for example, controls the network node associated with the direct pathto memory block M_(g,h). For example in FIG. 7, processor P03 708controls nodes L03 768 and T03 748, processor P21 714 controls nodes L21774 and T21 754. In a single instruction multiple data (SIMD) mode ofoperation, each direction associated with a load and store instructionfrom all the nodes involved in the operation provide the same directioncommand code. For example, a load from the east would be specified in abit field of a load instruction and that bit field portion of the loadinstruction would be the same for all load instructions in allprocessors involved in the operation. It is appreciated that differentexecution specific instruction operations such as different directionsof load or store operations may be specified among a group of executingnodes where the communication operations do not conflict. As anotherexample, in a specified group of processor nodes the non-communicationoriented bit field portions of the load instructions may be differentfor each processor node such that data from different memory addressesmay be fetched. When data is returned through the WAM network, it isloaded directly to the arithmetic unit of each processor that is doing aload operation, for example, receiving load data on load operand 1WAMXL1 1111 and load operand 2 WAMXL2 1112.

To associate an arithmetic operation with a load instruction, thelatency of the fetch through the WAM network must be accounted for. Forexample, with a single cycle allocated to address a memory block andobtain the data at the memory block output and a single cycle allocatedto transfer the fetched data across the network to a processor node, twocycles may be used for a data load operation.

Store operations follow a similar path with a store operand data at aspecified memory address is sent through the store WAMXS network to thememory based on the direction command in the store instruction. Thestore operand WAMXS 1113 and store address & store network opcode 1114are sent through the network to the desired memory block for storage.

FIG. 11B illustrates an example of a WAM processor system 1130. G×Hprocessors P_(0,0) 1132, P_(0,1) 1133, . . . , P_(G-1,H-1) 1134 areconnected to a Wings intelligence fetcher (WinF) 1136 through threeinstruction memory address lines 1137-1139. For example, instructionmemory address and control lines 1137-1139. The memory address andcontrol lines are similar to the WinF IMO address and controls 1105,WinF IM1 address and controls 1106, and WinF IM2 address and controls1107, respectively, as shown in the processor 1100 of FIG. 11A. TheWings intelligent fetcher 1136 fetches its instructions from the Wingsfetch instruction memory (WIM) 1140. The multiple processors connect todata memory through WAM networks, such as two WAMXL load networks,WAMXLA 1142 and WAMXLB 1143, and a WAMXS store network WAMXS1 1144. Withtwo WAM load networks, either multi-port memories or two memory blocksper associated processor node may be used, for example. In FIG. 11 B theWAM processor system 1130 uses two memory blocks per associatedprocessor node. For example, there are two memory blocks, MA_(0,0) 1146and M13 _(0,0) 1147 associated with processor node P_(0,0) 1132.

FIG. 11C illustrates a WAM16 processor subsystem 1150 with a set ofprocessor nodes 1152, a WAM16S/WAM16L combined network 1153, a first setof memories 1154, and a second set of memories 1155 in accordance withthe present invention. The WAM16S/WAM16L combined network 1153 is madeup of a WAM16S network, such as WAM16S network 400 of FIG. 4A, and aWAM16L network, such as WAM16L network 500. The WAM16S/WAM16L combinednetwork 1153 is used for connecting processor nodes 1152 to the firstset of memories 1154. The second set of memories 1155 connects locallyto the processor nodes 1152. With this organization simultaneous dualmemory loads to the processor nodes 1152 can be supported. Fourprocessor nodes 1156-1159 are illustrated in FIG. 11C that are part of alarger sixteen processor node network, such as illustrated in FIGS. 4Aand 5A, for example. For store operations processor nodes 1156-1159 senddata to the Rxx nodes 1160-1163. For example, processor node P01 1157sends data to R00 1160, R01 1161, and R02 1162. The Rxx nodes 1160-1163connect to Sxx nodes 1164-1167 and other nodes in the WAM16S/WAM16Lcombined network 1153. The Sxx nodes 1164-1167 connect to memories1168-1171, respectively. Though a single block of memory is shown foreach of the memories 1168-1171, it is appreciated that the memories1168-1171 may be partitioned into multiple memory blocks each accessibleby use of addressing ranges. The desired memory block may be specifiedthrough the memory address that is associated with the data being sentthrough the network for storage in memory.

For network load operations, a processor node initiates a network loadoperation by sending a data fetch address and network opcode through thenetwork to the desired memory. The addressed memory fetches data at thespecified address and send the data through the WAM16S/WAM16L combinednetwork 1153 back to the processor node that initiated the network loadoperation, such as one of the processor nodes 1156-1159. The memories1168-1171 are connected to Txx nodes 1172-1175. For example, memory M001168 sends data to T00 1172, T01 1173, and T03 1175. The Txx nodes1172-1175 connect to Lxx nodes 1176-1179 and other nodes in theWAM16S/WAM16L combined network 1153. The Lxx nodes 1176-1179 connect tothe processor nodes 1156-1159, respectively.

For local load operations, a processor node initiates a local loadoperation by sending a data fetch address directly to the local memoryassociated with the processor node. The local memory accesses the dataand provides it locally to the requesting processor node. For example,processor nodes 1156-1159 may load data from local memories 1180-1183,respectively.

Depending upon the application and processor cycle time, it is possibleto store through a WAMXS network into memory in a single cycle and toload data from a memory through a WAMXL network into a processor also ina single cycle. Such performance may be appropriate for low powerapplications, for example. For this type of situation, a softwarepipeline of storing and loading may be easily obtained providing asingle cycle throughput for communicating data between processor nodesfor any node in the system.

FIG. 11D illustrates a combined network node 1185 that combines a WAM16Snode 1186 and a WAM16L 1187 node into a single node 1188. The singlenode 1188 illustrates the functional aspect of the WAM nodes. The WAM16Snode 1186 and WAM16L node 1187 operate under control signal inputsprovided by decoder 1189 and 1190, respectively. The outputs of thedecoders 1189 and 1190 are represented by the binary state lists 1191and 1192, respectively. The decoders 1189 and 1190 receive controlsignals SNOp 1193 and LNOp 1194, respectively. For simple directionalpath control for the data through the networks, the WAM16S node 1186 andWAM16L 1187 node may be multiplexers selecting the path according to thebinary state indicated in the node diagram. In an alternativeembodiment, the control signals SNOp 1193 and LNOp 1194 are useddirectly without need for a decoder. The controls signals SNOp 1193 andLNOp 1194 connect directly to binary multiplexer control inputs that areused for controlling the multiplexers. In another alternativeembodiment, the decoders 1189 and 1190 in select modes of operation passthe control signals through the decoders and providing no additionaldecoding function. For additional functions of the nodes 1186 and 1187,the nodes 1186 and 1187 may provide different operations on data cominginto the nodes, as may be required by an application. These additionalfunctions may be specified by a more complex decoder implementation ofdecoders 1189 and 1190 and an expansion of the control signals SNOp 1193and LNOp 1194. For example, operations on individual data such as shiftoperations may be specified and more complex operations on multipleinput paths, such as compare and addition operations and the like mayalso be specified.

FIG. 12A illustrates Wings processor node 1200 made up of an executionnode 1202 and a memory node 1204 in accordance with an embodiment of thepresent invention. The split organization of the processor node 1200allows the execution node 1202 to be placed at the data input and outputconnections of a WAM store network, such as the WAM16S network 400 ofFIG. 4A and a WAM load network, such as the WAM16L network 500 of FIG.5A. The split organization of the processor node 1200 also allows thememory node 1204 to be placed at the data input and output connectionsof a WAM store network and a WAM load network. A WAM store networkcombined with a WAM load network is represented by network 1206.

The execution node 1202 receives arithmetic/logic instructions over anIM1 instruction bus 1212 connected to an arithmetic decode and executionunit 1214. The arithmetic/logic (AL) instructions each have a formatsuch as the AL instruction format 1020 of FIG. 10B. The received ALinstruction is decoded and executed using source operands XL1DataOut1215 and XL2DataOut 1216 supplied from the network 1206. The arithmeticdecode and execute unit 1214 generates a result XSDataIn 1217 that issent to the network 1206. The AL instruction itself contains no sourceor target operand information.

The memory node 1204 receives store instructions over an IM2 instructionbus 1222 connected to a store decode and execute unit 1224. The storeinstructions each have a format such as the store instruction format1040 of FIG. 10C. The received store instruction is decoded and executedgenerating address lines 1225 that are supplied to memory 1226 andcontrols (XScntls) 1228 supplied to the network 1206. XSDataIn 1217follows the data path of a WAM store network that is part of the network1206 and outputs a XSDataOut 1218. The XSDataOut 1218 is connected tothe memory 1226 and written to memory 1226 when the store instruction isexecuted. The Xscntls 1228 provide multiplexer control signals to thestore portion of the network 1206, such as the WAM16S network 400 ofFIG. 4A, such as multiplexer node 1186 of FIG. 11D.

The memory node 1204 further receives load instructions over an IM0instruction bus 1232 connected to a load decode and execute unit 1234.The load instructions each have a format such as the load instructionformat 1060 of FIG. 10D. The received load instruction is decoded andexecuted generating load address lines to be output to the memory 1226.For dual load instructions, for example, address lines 1235 and 1236 aregenerated. Associated with the generated address lines 1235 and 1236 arecorresponding control lines XL1cntls 1237 and XL2cntls 1238,respectively. The XL1cntls 1237 and XL2cntls 1238 provide multiplexercontrol signals to the load portion of the network 1206, such as havingtwo WAM16L networks 500 of FIG. 5A and using a multiplexer node, suchas, multiplexer node 1187 of FIG. 11D for each node of the loadnetworks. The two load address lines 1235 and 1236 cause two dataoperands to be read from memory 1226 and output on XL1DataIn 1240 andXL2DataIn 1241 that are connected to the network 1206. The XL1DataIn1240 and XL2DataIn 1241 follow a WAM load network path to reach theXL1DataOut 1215 and XLO2DataOut 1216, respectively.

By placing the load and store execute units 1234 and 1224 in closeproximity to the memory 1226, the load address lines 1235 and 1236 andstore address lines 1225 do not have to pass through the network 1206.The control signals XL1cntls 1237, XL2cntls 1238, and XScntls 1228 areused for multiplexer control in network 1206.

FIG. 12B illustrates processor node 1250 made up of an execution node1252 and a memory node 1254 in accordance with an embodiment of thepresent invention. The execution node 1252 does not have a decoder andreceives decoded arithmetic/logic instruction control signals 1256 froman external instruction decoder such as decoder 1260. The memory node1254 does not have a decoder and receives decoded store and loadinstructions control signals 1257 and 1258, respectively, from anexternal instruction decoder such as decoder 1260. The store and loadinstruction control signals 1257 and 1258 are received in port latch andcontrol units 1262 and 1264, respectively. The port latch and controlunit 1262 supplies the XScntls 1266 to a network 1270. The port latchand control unit 1264 supplies the XL1cntls 1268 and XL2cntls 1269 tothe network 1270. The port latch and control unit 1262 supplies thewrite address 1272 to a multiport memory, such as memory 1276. Datareceived from the network on XSDataOut 1282 is stored in the multiportmemory 1276. The port latch and control unit 1264 supplies the readaddresses 1273 and 1274 to a multiport memory, such as the memory 1276to access two data values. The data values are supplied to the networkon XL1DataIn 1283 and XL2DataIn 1284. In this fashion, singleinstructions, such as instructions 1285 may be separately decoded anduse the features and advantages of the present invention.

FIG. 13 illustrates a memory node to T node subsystem 1300 in accordancewith the present invention. The subsystem 1300 comprises a memory nodeM22 735 coupled to three T nodes, T12 751, T22 755, and T32 759 asrepresentative nodes from the WAM16L network 700 of FIG. 7. The memorynode M22 735 is coupled to T12 751 with a first bus 1302, coupled to T22755 with a second bus 1303, and coupled to T32 759 with a third bus1304. The memory node M22 735 may separately control the three busses1302-1304 to pass different information to each T node in parallel. Thememory node M22 735 may also control the three buses 1302-1304 to passthe same information to each T node in parallel, such as may be requiredfor a broadcast type of operation or the like. The memory node M22 735may also pass different combinations of information or no information onthe three buses. The information passed on the buses is generallyinformation that is stored in memory on the memory node M22 735.

FIG. 14 illustrates an exemplary WAM16S network 1400 in a physicallayout form of the WAM sixteen processor store (WAM16S) network 400 ofFIG. 4A in accordance with the present invention. The processors405-420, memory blocks 425-440, and network R nodes 445-460 and S nodes465-480 are distributed according to a G×H matrix where G=H=4. Eachprocessor P_(g,h), memory block M_(g,h) ,and internal nodes of thenetwork are labeled in a row g by column h format where gε{0,1,2,3} andhε{0,1,2,3}. The processors P_(g,h) 405-420 and first stage nodesR_(g,h) 445-460 are separately coupled across each row g. The firststage nodes R_(g,h) 445-460 and the second stage nodes S_(g,h) 465-480are separately coupled across each columns h. In an exemplaryimplementation, the processors P_(g,h) 405-420 and first stage nodesR_(g,h) 445-460 may be organized on one layer of a multi-layer siliconchip. A different layer of the chip may be utilized for the couplingbetween the first stage nodes R_(g,h) 445-460 and the second stage nodesS_(g,h) 465-480. The memory blocks 425-440 may be configured on the samelayer with the second stage nodes S_(g,h) 465-480 or on a differentlayer, such as the top layer of the chip, for example. In such anorganization, the memory blocks 425-440 may be overlaid upon theprocessors as shown in FIG. 4B.

FIG. 15 illustrates an exemplary WAM16L network 1500 physical layoutform of the alternative WAM16L network 700 of FIG. 7 in accordance withthe present invention. Processor nodes 705-720, memory nodes 725-740,and network nodes 745-760 and 765-780 are distributed according to a G×Hmatrix where G=H=4. Each processor node P_(g,h) , memory node M_(g,h),and internal nodes of the network are labeled in a row g by column hformat where gε{0,1,2,3} and hε{0,1,2,3}. A first set of nodes, such asmemory nodes M_(g,h) 725-740, for example, and a second set of nodesT_(g,h) 745-760 are separately coupled across each column h. The secondset of nodes T_(g,h) 745-760 and a third set of nodes L_(g,h) 765-780are separately coupled across each row g. The third set of nodes L_(g,h)765-780 are coupled to a fourth set of nodes, such as processor nodesP_(g),_(h) 705-720. In an exemplary implementation, the processorsP_(g),_(h) 705-720 and third set of nodes L_(g,h) 765-780 may beorganized on one layer of a multi-layer silicon chip. A different layerof the chip may be utilized for the coupling between the second set ofnodes T_(g,h) 745-760 and the third set of nodes L_(g,h) 765-780. Thememory nodes 725-740 may be configured on the same layer with the secondset of nodes T_(g,h) 745-760 or on a different layer, such as the toplayer of the chip. In such an organization the memory nodes 725-740 maybe overlaid upon the processors in a similar manner as shown in FIG. 4Bwith load paths utilized instead of the store paths shown in FIG. 4B.

FIG. 16A illustrates an exemplary combined network node 1600 thatcombines a WAM load node and a WAM store node into a combined node wherethe load and store nodes support expanded function capabilities inaccordance with the present invention. The combined node 1600illustrates functional capabilities of the WAM nodes. The R00 node 1602is similar to the WAM16S R00 node 445 of FIG. 4A and the T00 node 1604is similar to the WAM16L T00 node 745 of FIG. 7. Both the R00 node 1602and the T00 node 1604 are configured to operate in response to a controlsignal provided by decoder 1608 and 1610, respectively. The outputs ofthe decoders 1608 and 1610 may be represented by the binary state lists1612 and 1614, respectively. The decoders 1608 and 1610 receive controlsignals RNOp 1616 and TNOp 1618, respectively. For simple directionalpath control for the data through the networks, the R function (RFun)circuit 1603 and T function (TFun) circuit 1605 may be multiplexersselecting a path according to the binary state lists 1612 and 1614indicated in the node diagram. For example, the multiplexers within theRFun circuit 1603 or within the TFun circuit 1605 may be a singlemultiplexer as shown in FIG. 8A or organized as a set of multiplexers asshown in FIG. 8B.

In another embodiment, the “Sxx” nodes, such as the WAM16S S00 node 465of FIG. 4A, and the “Lxx” nodes, such as the WAM16L L00 node 765 of FIG.7, may be organized separately or combined in a similar manner to thecombined node 1600. The S00 node 465 and the L00 node 765 may beinternally organized as a set of multiplexers, for example as shown inFIG. 8B. In a network using such nodes, a memory node Mxx, coupled toits associated Sxx node, may be configured as a three port memory node.Similarly, a processor node Pxx, coupled to its associated Lxx node, maybe configured as having three input ports.

In alternative embodiments, the RFun circuit 1603 and the TFun circuit1605 may be multiplexers, function circuits, such as arithmetic or logiccircuits, or combinations of multiplexers and function circuits. Forexample, control signals RNOp 1616 and TNOp 1618 may be used directly tocontrol the RFun circuit 1603 and the TFun circuit 1605, respectively,without need for a decoder. The controls signals RNOp 1616 and TNOp 1618may be coupled directly to binary multiplexer control inputs, forexample, that are used for controlling multiplexers in the respectivefunction circuit. In another alternative embodiment, the decoders 1608and 1610, in select modes of operation, may pass the control signalsthrough the decoders and the decoders provide no additional decodingfunction. The nodes 1602 and 1604 may be configured to provide differentoperations on data coming into the nodes, as may be required by anapplication. These additional functions may be specified by a morecomplex decoder implementation of decoders 1608 and 1610 and anexpansion of the control signals RNOp 1616 and TNOp 1618. For example,the RFun circuit 1603 and the TFun circuit 1605 may be configured toprovide operations on individual data such as specifying shiftoperations or more complex operations on multiple input paths, such asmultiplication, multiplication and accumulation (MAC), compare, additionoperations, such as a three input addition for 1 to 3 adjacencynetworks, a five input addition for 1 to 5 adjacency networks, or thelike, complex number operations, or the like may also be specified. A 1to N adjacency network is described in more detail below. The R00 node1602 and the TOO node 1604 and their associated decoders may also beseparately placed.

FIG. 16B illustrates another alternative WAM network node 1650constructed using three sub-node units 1654-1656 comprising input andoutput interfaces and node function units (NodeFuns) 1658-1660,respectively. Since there are three NodeFuns 1658-1660 in thealternative WAM network node 1650, a decoder 1662 is configured todecode NodeOp input 1663 and generate three sets of control signals 1664to appropriately control the three NodeFuns 1658-1660. External inputs1668-1670 to the alternative WAM network node 1650 may be sent from aprocessor node, a previous node in the network, or from a memory node,for example. In one embodiment, input A 1668 may be selected by NodeFunA1658, input B 1669 may be selected by NodeFunB 1659, and input C 1670may be selected by NodeFunC 1660. In other embodiments, the inputs1668-1670 may be selected by the NodeFuns 1658-1660 in a different orderor in different combinations, such as inputs 1668-1670 selected in eachof the NodeFuns 1658-1660 and with different operations configured ineach of the NodeFun units. Each of the three NodeFuns 1658-1660 may beappropriately configured with a function as required or as selected fora particular implementation. Each output 1672-1674 of the alternativeWAM network node 1650 is separately sourced by its associated NodeFuns1658-1660, respectively.

FIG. 17 illustrates an exemplary configuration 1700 of the WAM16Snetwork of FIG. 4A combined with the alternative WAM16L network of FIG.7 in a physical layout form in accordance with the present invention.Processor nodes P00-P33 are combined with network nodes L00-L33 asP/LOO-P/L33 nodes 1705-1720, respectively. Memory nodes M00-M33 arecombined with network nodes S00-S33 as S/M00-S/M33 nodes 1725-1740,respectively. Network nodes R00-R33 are combined with network nodesT00-T33 as R/T00-R/T33 nodes 1745-1760, respectively. The R/T00-R/T33nodes 1745-1760 are patterned after the exemplary combined network node1600 of FIG. 16A. The P/L00-P/L33 nodes 1705-1720, the S/M00-S/M33 nodes1725-1740, and the R/T00-R/T33 nodes 1745-1760 are distributed accordingto a G×H matrix where G=H=4. Each processor node P_(g,h), memory nodeM_(g,h), and internal nodes of the network are labeled in a row g bycolumn h format where gε{0,1,2,3} and hε{0,1,2,3}. The S/M00-S/M33 nodes1725-1740 and the R/TOO-R/T33 nodes 1745-1760 are separately coupledacross each column h. The R/T00-R/T33 nodes 1745-1760 and theP/LOO-P/L33 nodes 1705-1720 are separately coupled across each row g. Inan exemplary implementation, the P/LOO-P/L33 nodes 1705-1720 may beorganized on a first layer of a multi-layer silicon chip. A differentlayer of the chip may be utilized for the coupling between theR/TOO-R/T33 nodes 1745-1760 and the P/LOO-P/L33 nodes 1705-1720. Thecoupling between the S/M00-S/M33 nodes 1725-1740 and the WTOO-R/T33nodes 1745-1760 may be configured on a third different layer. In anotherembodiment the S/M00-S/M33 nodes 1725-1740 may be configured on the toplayer of the chip. These and other layout configurations may be used tominimize wire length and implementation complexity. In such anorganization, the memory nodes may be overlaid upon the processors in asimilar manner as shown in FIG. 4B with load paths and store pathsincluded.

FIG. 18 illustrates a Wings array memory (WAM) twenty five processor(WAM25S) network 1800 for store (S) operations. The processor nodes 1804and memory nodes 1806 are organized in linear arrays and identifiedaccording to a G×H matrix where, in this example, G equals fiverepresenting the number of rows in the matrix and H equals fiverepresenting the number of columns. Each processor P_(g,h), memory nodeM_(g,h),and internal nodes of the two stage network 1808 are labeled ina row g by column h format where gε{0,1, . . . , G−1} and hε{0,1, . . ., H−1}. The processors are not directly connected to each other nor arethe memory blocks directly connected to any of the other memory blocks.

The two stage WAM network 1808 couples the processor nodes 1804 andmemory nodes 1806 for store operations. A first stage of R_(g,h) nodes1810 are labeled in a row g by column h matrix. A second stage ofS_(g,h) nodes 1812 are also labeled in a row g by column h matrix. Theprocessors P_(g,h) each have an output, the memory nodes M_(g,h) eachhave an input, and the R_(g,h) nodes and the S_(g,h) nodes each havethree inputs and an output. The processors P_(g,h), the memory blocksM_(g,h), the multiplexers R_(g,h), and the multiplexers S_(g,h) arelabeled in the figures as Pgh, Mgh, Rgh, and Sgh, respectively, for easeof notation and reference in the figures. The first stage of processorsP_(g,h) and R_(g,h) nodes are partitioned into groups by rows of theG=5×H=5 matrix. For example, in the g=0 row 1816, the outputs of theprocessors P00, P01, P02, P03, and PO4 are coupled to the inputs of theR00, R01, R02, R03, and R04 nodes. For the g=1 row 1818, the outputs ofthe processors P10, P11, P12, P13, and P14 are coupled to the inputs ofthe R10, R11, R12, R13, and R14 nodes. For the g=2 row 1820, the outputsof the processors P20, P21, P22, P23, and P24 are coupled to the inputsof the R20, R21, R22, R23, and R24 nodes. For the g=3 row 1822,processors P30, P31, P32, P33, and P34 are coupled to the R30, R31, R32,R33, and R34 nodes. For the g=4 row 1824, processors P40, P41, P42, P43,and P44 are coupled to the R40, R41, R42, R43, and R44 nodes.

In each group, the connections are made according to an adjacency ofnodes in a first dimension. For example, in the g=0 row 1816, P00 iscoupled to R00, R01, and R04. P01 is coupled to R00, R01, and R02. P02is coupled to R01, R02, and R03. P03 is coupled to R02, R03, and R04.P04 is coupled to R00, R03, and R04. Each processor in the g=1 row 1818,P10-P14, the g=2 row 1820 P20-P24, the g=3 row 1822 P30-P34, and g=4 row1824 P40-P44, are coupled to R nodes in a similar fashion as the g=0 row1816 according to the nodes adjacency in the rows.

The R_(g,h) nodes are coupled to the S_(g,h) nodes according to anadjacency of nodes in a second dimension. Each output of the S_(g,h)nodes is coupled to the input of their associated memory node at thesame row column position. A processor executing a store operation canwrite data to a single memory node or combinations of up to nine memorynodes from the memory array 1806. For example, processor P21 can storedata to memories in its coupled group of memory nodes, including M10,M20, M30, M11, M21, M31, M12, M22, and M32.

The adjacency of nodes is represented by a G×H matrix where the nodes ofthe matrix may be processors, arithmetic function units, memory nodes,multiplexers, sensors, or the like, generally, having nodes N_(g,h)where gε{0,1, . . . ,G-1} and hε{0,1, . . . ,H-1}. A connection network,such as the WAM25S network 1800 of FIG. 18, may be generalized as havinga first set of nodes, such as processor nodes P_(g,h), for example,coupled to a second set of nodes R_(g,h) which are coupled to a thirdset of nodes S_(g,h). The third set of nodes S_(g,h) then are coupled toa fourth set of nodes, such as memory nodes M_(g,h), for example.

The store connectivity of the nodes can be viewed as follows:

Output Coupled to an of Node input of the Nodes Where P_(g, h) R_(g, h),R_(g, h+1), and h + 1 wraps to 0 when h + 1 = H and R_(g, h−1) h − 1wraps to H − 1 when h − 1 = −1The R_(g,h) nodes are coupled to the S_(g,h) nodes as follows:

Output Coupled to an of Node input of the Nodes Where R_(g, h) S_(g, h),S_(g+1, h), and g + 1 wraps to 0 when g + 1 = G and S_(g−1, h) g − 1wraps to G − 1 when g − 1 = −1The nodes S_(g,h) nodes are coupled to the M_(g,h) nodes as follows:

Output of Node Connects to the input of the Node S_(g, h) M_(g, h)

A connectivity matrix A₄ for the connections between the processorsP_(g,h) and the R_(g,h) nodes in a row g=0, termed a 1 to 3 adjacencyfor notational purposes, for the WAM16S network of FIG. 4A is shown inTable 1. A “1” in a cell of the connectivity matrix indicates aconnection between the PEs and R nodes in a row. For example, three “1”spopulate P02 row, indicating that P02 connects to R01, R02, and R03 butnot to R00. Table 2 is a 4×4 identity matrix I₄.

TABLE 1 4 × 4 R00 R01 R02 R03 A₄ = P00 1 1 0 1 P01 1 1 1 0 P02 0 1 1 1P03 1 0 1 1

TABLE 2 I₄ = 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1

Tensor product algebra is used to describe the Wings networkconnectivity. Using tensor product notation, a tensor product of twomatrices Y₂ and I₄ is I₄

Y₂, where I₄ is the identity matrix and

$Y_{2} = {\begin{bmatrix}a & b \\c & d\end{bmatrix}\text{:}}$ $\left( {\begin{bmatrix}1 & 0 & 0 & 0 \\0 & 1 & 0 & 0 \\0 & 0 & 1 & 0 \\0 & 0 & 0 & 1\end{bmatrix} \otimes \begin{bmatrix}a & b \\c & d\end{bmatrix}} \right) = \begin{bmatrix}a & b & 0 & 0 & 0 & 0 & 0 & 0 \\c & d & 0 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & a & b & 0 & 0 & 0 & 0 \\0 & 0 & c & d & 0 & 0 & 0 & 0 \\0 & 0 & 0 & 0 & a & b & 0 & 0 \\0 & 0 & 0 & 0 & c & d & 0 & 0 \\0 & 0 & 0 & 0 & 0 & 0 & a & b \\0 & 0 & 0 & 0 & 0 & 0 & c & d\end{bmatrix}$Y₂

I₄, where I₄ is the identity matrix and

$Y_{2} = {\begin{bmatrix}a & b \\c & d\end{bmatrix}\text{:}}$ $\left( {\begin{bmatrix}a & b \\c & d\end{bmatrix} \otimes \begin{bmatrix}1 & 0 & 0 & 0 \\0 & 1 & 0 & 0 \\0 & 0 & 1 & 0 \\0 & 0 & 0 & 1\end{bmatrix}} \right) = \begin{bmatrix}a & 0 & 0 & 0 & b & 0 & 0 & 0 \\0 & a & 0 & 0 & 0 & b & 0 & 0 \\0 & 0 & a & 0 & 0 & 0 & b & 0 \\0 & 0 & 0 & a & 0 & 0 & 0 & b \\c & 0 & 0 & 0 & d & 0 & 0 & 0 \\0 & c & 0 & 0 & 0 & d & 0 & 0 \\0 & 0 & c & 0 & 0 & 0 & d & 0 \\0 & 0 & 0 & c & 0 & 0 & 0 & d\end{bmatrix}$

A number of useful properties of tensor products include a mixed productrule [(A

B)(C

D)=AC

BD], an associative property [A

(B

C)=(A

B)

C], and an identity property [I_(xy)=I_(x)

I_(y)].

The first stage of the WAM16S network 400 of FIG. 4A for the processornodes Pxx to the Rxx nodes may be represented by I₄

A₄ where

indicates the tensor product. The second stage of FIG. 4A for the Rxxnodes to the Sxx nodes may be represented by A₄

I₄. The combination of the two stages is given by:(I ₄

A ₄)(A ₄

I ₄)=A ₄

A ₄

The first stage of FIG. 7 for the memory nodes Mxx to the Txx nodes maybe represented by A₄

I₄. The second stage of FIG. 7 for the Txx nodes to the Lxx nodes may berepresented by I₄

A₄. The combination of the two stages is given by:(A ₄

I ₄)(I ₄

A ₄)=A ₄

A ₄

The combination of the store and load networks is given by:(A ₄

A ₄)(A ₄

A ₄)=(A ₄ *A ₄)

(A ₄ *A ₄)

For (A₄

A₄)(A₄

A₄) to represent a completely connected network, the matrix (A₄*A₄) mustbe all ones, otherwise a path is not connected. Using binary matrixmultiplication where multiplication of two elements is a logical ANDoperation and addition of two elements is a logical OR operation,(A₄*A₄) is:

${\begin{bmatrix}1 & 1 & 0 & 1 \\1 & 1 & 1 & 0 \\0 & 1 & 1 & 1 \\1 & 0 & 1 & 1\end{bmatrix}*\begin{bmatrix}1 & 1 & 0 & 1 \\1 & 1 & 1 & 0 \\0 & 1 & 1 & 1 \\1 & 0 & 1 & 1\end{bmatrix}} = \begin{bmatrix}1 & 1 & 1 & 1 \\1 & 1 & 1 & 1 \\1 & 1 & 1 & 1 \\1 & 1 & 1 & 1\end{bmatrix}$

Thus the combination of the WAM16S network 400 of FIG. 4A and the loadWAM16L network 700 of FIG. 7 is a completely connected network with adiameter of 2.

A connectivity matrix A₅ for the connections between the processorsP_(g,h) and the R_(g,h) nodes in a row g=0, termed a 1 to 3 adjacencyfor notational purposes, for the WAM25S network of FIG. 18 is shown inTable 3.

TABLE 3 5 × 5 R00 R01 R02 R03 R04 A₅ = P00 1 1 0 0 1 P01 1 1 1 0 0 P02 01 1 1 0 P03 0 0 1 1 1 P04 1 0 0 1 1Table 4 is a 5×5 identity matrix I₅.

TABLE 4 I₅ = 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1

The WAM25S network of FIG. 18 having two stages may be represented by:(I ₅

A ₅)(A ₅

I ₅)=A ₅

A ₅

A corresponding load WAM25L network having two stages may be representedby:(A ₅

I ₅)(I ₅

A ₅)=A ₅

A ₅

The combination of the store and load networks may be represented by:(A ₅

A ₅)(A ₅

A ₅)=(A ₅ *A ₅)

(A ₅ *A ₅)

For (A₅

A₅)(A₅

A₅) to represent a completely connected network, the matrix (A₅*A₅) mustbe all ones, otherwise a path is not connected. Using binary matrixmultiplication where multiplication of two elements is a logical ANDoperation and addition of two elements is a logical OR operation,(A₅*A₅) is:

${\begin{bmatrix}1 & 1 & 0 & 0 & 1 \\1 & 1 & 1 & 0 & 0 \\0 & 1 & 1 & 1 & 0 \\0 & 0 & 1 & 1 & 1 \\1 & 0 & 0 & 1 & 1\end{bmatrix}*\begin{bmatrix}1 & 1 & 0 & 0 & 1 \\1 & 1 & 1 & 0 & 0 \\0 & 1 & 1 & 1 & 0 \\0 & 0 & 1 & 1 & 1 \\1 & 0 & 0 & 1 & 1\end{bmatrix}} = \begin{bmatrix}1 & 1 & 1 & 1 & 1 \\1 & 1 & 1 & 1 & 1 \\1 & 1 & 1 & 1 & 1 \\1 & 1 & 1 & 1 & 1 \\1 & 1 & 1 & 1 & 1\end{bmatrix}$

Thus, the combination of the WAM25S network 1800 of FIG. 18 and the loadWAM16L network is a completely connected network with a diameter of 2.

The 4×4 WAM16S/WAM16L combined network having the connectivity network670 of FIG. 6C has a diameter of 2 between any two processor nodes orbetween any two memory nodes in the combined network. The 5×5WAM25S/WAM25L combined network also has a diameter of 2 between any twoprocessor nodes or between any two memory nodes in the combined network.

FIG. 9A illustrates a WAM64 tore network having sixty-four processorelements (Ps), a network for store operations, having sixty-four firststage multiplexers (Rs), sixty-four second stage multiplexers (Ss),sixty-four third stage multiplexers (Vs), and sixty-four memory elements(Ms). The WAM64 store network is based on a G×H×K 3-dimensional cubeorganization. The first stage connections of FIG. 9A, for connectionsPxx to Rxx in a first dimension, where the Rxx nodes are multiplexernodes, may be represented by (I₄

(I₄

A₄)). The second stage of FIG. 9A, for connections Rxx to Sxx in asecond dimension, where the Sxx nodes are multiplexer nodes, may berepresented by (I₄

(A₄

I₄)). The third stage of FIG. 9A, for connections Sxx to Vxx in a thirddimension, where the nodes Vxx are multiplexer nodes, may be representedby ((A₄

I₄)

I₄). The combination of the three stages may be represented by:(I ₄

(I ₄

A ₄))(I ₄

(A ₄

I ₄))((A ₄

I ₄)

I ₄)=(A ₄

(A ₄

A ₄))Since, without consideration of the direction of the connection paths,the connections for the load network are generally the same as theconnections for the store network, the connection matrix for the loadnetwork may be represented by (A₄

(A₄

A₄)). Thus, the combination of the store and load networks may berepresented by:(A ₄

(A ₄

A ₄))(A₄

(A ₄

A ₄))=A ₄(A ₄)

(A ₄(A ₄)

A ₄(A ₄))

For (A₄

(A₄

A₄))(A₄

(A₄

₄)) to represent a completely connected network, the matrix A₄(A₄) mustbe all ones, otherwise a path is not connected. As shown above, thematrix A₄(A₄) has been shown to be all ones. Thus, the WAM store networkof FIG. 9A combined with a WAM load network of equivalent organizationis a completely connected network.

FIG. 19A illustrates a representative processor to memory path in aWings array memory (WAM) forty nine processor (WAM49S) network 1900 forstore (S) operations in accordance with the present invention. Theprocessor nodes and memory nodes are identified according to a G=7×H=7matrix where G equals seven representing the number of rows in thematrix and H equals seven representing the number of columns. The nodesare labeled in a row g by column h format where gε{0,1, . . . ,G-1} andhε{0,1, . . . ,H-1}. The processors are not directly connected to eachother nor are the memory blocks directly connected to any of the othermemory blocks.

The concept of adjacency is extended in a Wings array system. In astandard four neighborhood N×N mesh or torus, a P_(row,column)(P_(r,c))node is adjacent to nodes P_(r,c−1) and P_(r,c+1) in a first dimension.The P_(r,c+1) node is adjacent to the nodes P_(r,c+2) and P_(r,c) in thefirst dimension. The P_(r,c−1) node is adjacent to the nodes P_(r,c−2)and P_(r,c) in the first dimension. Couplings of the nodes at the edgesof a mesh may be implemented in an application specific manner.Wraparound couplings between nodes at the edges of a torus are describedin further detail below. Couplings between nodes in a first stage of aWings array system are made according to a double adjacency of nodes ina first dimension. In the first stage, a double adjacency of nodes in afirst dimension is defined for a P_(r,c) node to be coupled to nodesR_(r,c−2), R_(r,c−1), R_(r,c), R_(r,c+1), and R_(r,c+2). For example,the representative P22 node to memory path for the first stage beginswith the P22 node coupled to node R20 over path 1902, to node R21 overpath 1903, to node R22 over path 1904, to node R23 over path 1905, andto node R24 over path 1906. Couplings between nodes in a second stage ofthe Wings array system are made according to a double adjacency of nodesin a second dimension. In the second stage, a double adjacency of nodesin a second dimension is defined for an R_(r,c) node to be coupled tonodes S_(r−2,c), S_(r−1,c), S_(r,c), S_(r+1,c), and S_(r+2,c). Forexample, in the second stage, the R22 node is coupled to node S02 overpath 1912, to node S12 over path 1913, to node S22 over path 1914, tonode S32 over path 1915, and to node S42 over path 1916. In a Wingsarray memory network, a processor node executing a store operation canwrite data to a single memory node or to combinations of up to twentyfive memory nodes.

The double adjacency of nodes is represented in a G×H matrix where thenodes of the matrix may be processors, arithmetic function units, memorynodes, multiplexers, sensors, or the like, generally, having nodesN_(g,h) where gε{0,1, . . . ,G-1} and hε{0,1, . . . ,H-1}. FIG. 19Billustrates a general form of a double adjacency store path 1920selected from the WAM49S network 1900 of FIG. 19A in accordance with thepresent invention. The store path begins at P_(g,h) 1922 connecting in afirst stage 1923 of a WAM49S network to five R nodes 1924-1928. The fiveR nodes 1924-1928 connect in a second stage 1930 of the WAM49S networkto twenty five S nodes that each connect directly to a correspondingmemory node in the twenty five memory nodes group 1933.

The adjacent connections are as follows:

Output Coupled to an of Node input of the Nodes Where P_(g, h) R_(g, h),R_(g, h+1), R_(g, h+2), h + 1 wraps to 0 when h + 1 = H, R_(g, h−1), andR_(g, h−2) h + 2 wraps to 0 when h + 2 = H, h + 2 wraps to 1 when h + 2= H + 1 and h − 1 wraps to H − 1 when h − 1 = −1, h − 2 wraps to H − 1when h − 2 = −1, h − 2 wraps to H − 2 when h − 2 = −2The R_(g,h) nodes are coupled to the S_(g,h) nodes as follows:

Output Coupled to an of Node input of the Nodes Where R_(g, h) S_(g, h),S_(g+1, h), S_(g+2, h), g + 1 wraps to 0 when g + 1 = G, S_(g−1, h), andS_(g−2, h) g + 2 wraps to 0 when g + 2 = G, g + 2 wraps to 1 when g + 2= G + 1 and g − 1 wraps to G − 1 when g − 1 = −1, g − 2 wraps to G − 1when g − 2 = −1, g − 2 wraps to G − 2 when g − 2 = −2The nodes S_(g,h) nodes are coupled to the M_(g,h) nodes as follows:

Output of Node Connects to the input of the Node S_(g, h) M_(g, h)

FIG. 19C illustrates an exemplary double adjacency store path 1950selected from the WAM49S network 1900. The store path 1950 begins at P221952. This store path 1950 is formed by substituting g=2 and h=2 in thesubscripted notation of the general form of the double adjacency storepath 1920 in FIG. 19B. For example, processor P22 can store data tomemories in its coupled group of twenty five memory nodes, includingM00, M10, M20, M30, M40, M01, M11, M21, M31, M41, M02, M12, M22, M32,M42, M03, M13, M23, M33, M43, M04, M14, M24, M34, and M44.

A connectivity matrix A₇ for the connections between the nodes P_(g,h)and the nodes R_(g,h) in a row g=0, termed a 1 to 5 double adjacency fornotational purposes, for the WAM49S network of FIG. 19A is shown inTable 5.

TABLE 5 7 × 7 R00 R01 R02 R03 R04 R05 R06 A₇ = P00 1 1 1 0 0 1 1 P01 1 11 1 0 0 1 P02 1 1 1 1 1 0 0 P03 0 1 1 1 1 1 0 P04 0 0 1 1 1 1 1 P05 1 00 1 1 1 1 P06 1 1 0 0 1 1 1Table 6 is a 7×7 identity matrix I₇.

TABLE 6 I₇ = 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 00 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1

The WAM49S network of FIG. 19A having two stages may be represented by:(I ₇

A ₇)(A ₇

I ₇)=A ₇

A ₇A corresponding load WAM49L network with two stages may be representedby:(A ₇

I ₇)(I ₇

A ₇)=A ₇

A ₇

The combination of the store and load networks is given by:(A ₇

A ₇)(A ₇

A ₇)=(A ₇ *A ₇)

(A ₇ *A ₇)

For (A₇

A₇)(A₇

A₇) to represent a completely connected network, the matrix (A₇*A₇) mustbe all ones, otherwise a path is not connected. Using binary matrixmultiplication where multiplication of two elements is a logical ANDoperation and addition of two elements is a logical OR operation,(A₇*A₇) is:

${\begin{bmatrix}1 & 1 & 1 & 0 & 0 & 1 & 1 \\1 & 1 & 1 & 1 & 0 & 0 & 1 \\1 & 1 & 1 & 1 & 1 & 0 & 0 \\0 & 1 & 1 & 1 & 1 & 1 & 0 \\0 & 0 & 1 & 1 & 1 & 1 & 1 \\1 & 0 & 0 & 1 & 1 & 1 & 1 \\1 & 1 & 0 & 0 & 1 & 1 & 1\end{bmatrix}*\begin{bmatrix}1 & 1 & 1 & 0 & 0 & 1 & 1 \\1 & 1 & 1 & 1 & 0 & 0 & 1 \\1 & 1 & 1 & 1 & 1 & 0 & 0 \\0 & 1 & 1 & 1 & 1 & 1 & 0 \\0 & 0 & 1 & 1 & 1 & 1 & 1 \\1 & 0 & 0 & 1 & 1 & 1 & 1 \\1 & 1 & 0 & 0 & 1 & 1 & 1\end{bmatrix}} = \begin{bmatrix}1 & 1 & 1 & 1 & 1 & 1 & 1 \\1 & 1 & 1 & 1 & 1 & 1 & 1 \\1 & 1 & 1 & 1 & 1 & 1 & 1 \\1 & 1 & 1 & 1 & 1 & 1 & 1 \\1 & 1 & 1 & 1 & 1 & 1 & 1 \\1 & 1 & 1 & 1 & 1 & 1 & 1 \\1 & 1 & 1 & 1 & 1 & 1 & 1\end{bmatrix}$

Thus, the combination of the WAM49S network 1900 of FIG. 19A and thecorresponding load WAM49L network is a completely connected network witha diameter of 2.

A connectivity matrix A₉ for the connections between the processorsP_(g,h) and the R_(g,h) nodes in a row g=0, termed a 1 to 5 doubleadjacency for notational purposes, for a 9×9 WAM81S network is shown inTable 7.

TABLE 7 9 × 9 R00 R01 R02 R03 R04 R05 R06 R07 R08 A₉ = P00 1 1 1 0 0 0 01 1 P01 1 1 1 1 0 0 0 0 1 P02 1 1 1 1 1 0 0 0 0 P03 0 1 1 1 1 1 0 0 0P04 0 0 1 1 1 1 1 0 0 P05 0 0 0 1 1 1 1 1 0 P06 0 0 0 0 1 1 1 1 1 P07 10 0 0 0 1 1 1 1 P08 1 1 0 0 0 0 1 1 1Table 8 is a 9×9 identity matrix I₉.

TABLE 8 I₉ = 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 01 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 00 0 0 0 1 0 0 0 0 0 0 0 0 0 1

The WAM81S network having two stages may be represented by:(I ₉

A ₉)(A ₉

I ₉)=A ₉

A ₉

A WAM81L network two stages may be represented by:(A ₉

I ₉)(I ₉

A ₉)=A ₉

A ₉

The combination of the store and load networks may be represented by:(A ₉

A ₉)(A ₉

A ₉)=(A ₉ *A ₉)

(A ₉ *A ₉)

For (A₉

A₉)(A₉

A₉) to represent a completely connected network, the matrix (A₉*A₉) mustbe all ones, otherwise a path is not connected. Using binary matrixmultiplication where multiplication of two elements is a logical ANDoperation and addition of two elements is a logical OR operation,(A₉*A₉) is:

$\begin{bmatrix}1 & 1 & 1 & 0 & 0 & 0 & 0 & 1 & 1 \\1 & 1 & 1 & 1 & 0 & 0 & 0 & 0 & 1 \\1 & 1 & 1 & 1 & 1 & 0 & 0 & 0 & 0 \\0 & 1 & 1 & 1 & 1 & 1 & 0 & 0 & 0 \\0 & 0 & 1 & 1 & 1 & 1 & 1 & 0 & 0 \\0 & 0 & 0 & 1 & 1 & 1 & 1 & 1 & 0 \\0 & 0 & 0 & 0 & 1 & 1 & 1 & 1 & 1 \\1 & 0 & 0 & 0 & 0 & 1 & 1 & 1 & 1 \\1 & 1 & 0 & 0 & 0 & 0 & 1 & 1 & 1\end{bmatrix}*{\quad{\quad{\begin{bmatrix}1 & 1 & 1 & 0 & 0 & 0 & 0 & 1 & 1 \\1 & 1 & 1 & 1 & 0 & 0 & 0 & 0 & 1 \\1 & 1 & 1 & 1 & 1 & 0 & 0 & 0 & 0 \\0 & 1 & 1 & 1 & 1 & 1 & 0 & 0 & 0 \\0 & 0 & 1 & 1 & 1 & 1 & 1 & 0 & 0 \\0 & 0 & 0 & 1 & 1 & 1 & 1 & 1 & 0 \\0 & 0 & 0 & 0 & 1 & 1 & 1 & 1 & 1 \\1 & 0 & 0 & 0 & 0 & 1 & 1 & 1 & 1 \\1 & 1 & 0 & 0 & 0 & 0 & 1 & 1 & 1\end{bmatrix} = \begin{bmatrix}1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1\end{bmatrix}}}}$

The 9×9 WAM81S network having the 1 to 5 double adjacency connectivitynetwork of Table 7 when combined with a 9×9 WAM81L network has adiameter of 2 between any two processor nodes or between any two memorynodes in the combined network. Using a similar process, as describedabove, a 1 to 7 triple adjacency connectivity network may be constructedand used to show that a 7×7 network is configurable for a diameter ofone and a network up to a 13×13 network is configurable using load andstore communications for a diameter of two. Couplings between nodes in afirst stage of a Wings array system are made according to a tripleadjacency of nodes in a first dimension. In the first stage, a tripleadjacency of nodes in a first dimension is defined for a P_(r,c) node tobe coupled to nodes R_(r,c−3), R_(r,c−2), R_(r,c−1), R_(r,c), R_(r,c+1),R_(r,c+2), and R_(r,c+3). Couplings between nodes in a second stage ofthe Wings array system are made according to a triple adjacency of nodesin a second dimension. In the second stage, a triple adjacency of nodesin a second dimension is defined for a R_(r,c) node to be coupled tonodes S_(r−3,c), S_(r−2,c), S_(r−1,c), S_(r,c), S_(r+1,c), S_(r+2,c),and S_(r+3,c). Also, using a similar process, as described above, a 1 to9 quadruple adjacency connectivity network may be constructed and usedto show that a 9×9 network is configurable for a diameter of one and anetwork up to a 17×17 network is configurable using load and storecommunications for a diameter of two.

In general, couplings between nodes in a first stage of a Wings arraysystem are made according to an N-level adjacency of nodes in a firstdimension of a G×H matrix of nodes, where G≧N and H≧N. In the firststage, an N-level adjacency of nodes in a first dimension is defined fora P_(r,c) node to be coupled to nodes R_(r,c−└N/2┘), . . . , R_(r,c−2),R_(4,c−1), R_(r,c), R_(r,c+1), R_(r,c+2), . . . R_(r,c+└N/2┘), where Nis a positive odd integer and └N/2┘ is the floor of N/2 which is thelargest integer less than N/2 since N is odd. Couplings between nodes ina second stage of the Wings array system are made according to anN-level adjacency of nodes in a second dimension of the G×H matrix ofnodes, where G≧N and H≧N. In the second stage, an N-level adjacency ofnodes in a second dimension is defined for an R_(r,c) node to be coupledto nodesS_(r−└N/2┘,c), . . . , S_(r−2,c), S_(r−1,c), S_(r,c), S_(r+1,c),S_(r+2,c), . . . , S_(r+└N/2┘,c).

It is noted that other network configurations may be constructed usingthe principles of the present invention, such as having mixed levels ofadjacency of connections in different dimensions of communication. Forexample, a network may be constructed having a 1 to 3 single adjacencyof connections in a first dimension and a 1 to 5 double adjacency ofconnections in a second dimension. The choice of whether to use the samelevel of adjacency of connections in each dimension or a combination oflevels of adjacency of connections in different dimensions may be basedon an application requirement.

A listing of a number of network adjacency organizations using the sameadjacency in each dimension and associated properties is shown in Table9.

TABLE 9 2D Network 2D Network 3D Network Adjacency configurable forconfigurable for configurable for Connections a diameter of 1 a diameterof 2 a diameter of 2 1 to 3 Single 3 × 3 Up to 5 × 5 5 × 5 × 5 Adjacency1 to 5 Double 5 × 5 Up to 9 × 9 9 × 9 × 9 Adjacency 1 to 7 Triple 7 × 7Up to 13 × 13 13 × 13 × 13 Adjacency 1 to 9 9 × 9 Up to 17 × 17 17 × 17× 17 Quadruple Adjacency

Neural network models may provide insights into techniques for solvingdifficult computer system problems. For example, neural networksgenerally require highly parallel computations, a high level ofconnectivity among processing and memory nodes, efficient communicationbetween nodes, and cooperative computing to support learning andartificial intelligence capabilities. The Wings network andcomputational architecture provides a scalable massively parallelapproach that exploits storage and processing in the connections betweencomputational nodes. By using a scalable WAM network using load andstore instructions for communications, it may be possible to demonstratethat intelligence is not just a result of computation, but that thecouplings between nodes and the information that resides in suchcouplings plays an equal if not more important role in definingintelligence. Also, a Wings network system supporting neural processingmay be switched to more standard forms of parallel computation therebyproviding a unique paradigm that combines neural with standardcomputational techniques.

A 2-dimensional (2D) Wings neural network (2DWNN) processor is definedas a 2D G×H network of neurons, each neuron having an N×N array ofsynaptic weight values stored in coupled memory nodes, where G≧N, H≧N,and N is determined from a 1 to N adjacency of connections used in theG×H network. A 3-dimensional (3D) Wings neural network processor isdefined as a 3D G×H×K network of neurons, each neuron with an N×N×Narray of synaptic weight values stored in coupled memory nodes, whereG≧N, H≧N, K≧N, and N is determined from a 1 to N adjacency ofconnections used in the G×H×K network. A virtual neural network isdefined for each neuron with an M×M×M array of synaptic weight valuesstored in the coupled memory nodes, where M is greater than the Ndetermined from the 1 to N adjacency of connections used in the network.For the 2DWNN with a 1 to 3 adjacency of connections, the neuronprocessors are configured to operate according to:

-   P_(g,h)=F(W_((g,h),(g−1,h−1))*P_(g−1,h−1)+W_((g,h),(g,h−1))*P_(g,h−1)+W_((g,h),(g+1,h−1))*P_(g+1,h−1)+W_((g,h),(g−1,h))*P_(g−1,h)+W_((g,h),(g,h))*P_(g,h)+W_((g,h),(g+1,h))*P_(g+1,h)+W_((g,h)(g−1,h+1))*P_(g−1,h+1)+W_((g,h),(g,h+1))*P_(g,h+1)+W_((g,h),(g+1,h+1))*P_(g+1,h+1)    ), where W _((x),(y)) is interpreted to mean the weight of a    connection from neuron y to neuron x, for example, the weight    W_((g,h),(g+1,h−1)) is interpreted to mean the weight of a    connection from neuron (g+1,h−1) to neuron g,h. The neuron processor    P nodes apply a function F, which for a neural network may take the    form of a sigmoid function of the received input. The P_(g,h) neuron    output is applied to a coupled store network to be communicated to a    corresponding memory node M_(g,h) in the N×N array of synaptic    weight values.

An exemplary 2D neural network may be implemented based on the exemplaryconfiguration 1700 of FIG. 17 which combined the WAM16S network of FIG.4A with the alternative WAM16L network of FIG. 7. The Wings neuralnetworks are configured with internal nodes having similar capabilitiesas described for the combined network node 1650 with expanded functionsof FIG. 16B. FIG. 20A illustrates a load path 2000 to a neuron processorPgh 2010 in accordance with the present invention. The L and T nodescomprise arithmetic functions as shown in FIGS. 20B-20E and described inmore detail below. Each memory node supplies a current Pgh node valueand an associated plurality of weight values to the T nodes in the loadnetwork, first stage 2004. Based on the 1 to 3 adjacency of connections,memory nodes 2014-2016 provide current P node values and weight valuesto Tg(h−1) node 2024, memory nodes 2017-2019 provide current P nodevalues and weight values to Tgh node 2025, and memory nodes 2020-2022provide current P node values and weight values to Tg(h+1) node 2026.The T nodes 2024-2026 multiply the current P node values with thereceived weight value and provide a 3 to 1 summation of the multipliedvalues to the Lgh node 2008 in a second stage 2006. The L nodes providea summation of weighted neuron values to the neuron processor Pgh 2010to generate the next neuron value. The newly generated neuron value iscommunicated back to the memory nodes using a WAM store network. It isnoted that registers, buffers, queues and the like which may be requiredfor a particular implementation are not shown for clarity ofillustrating the inventive concepts.

FIGS. 20B, 20C, and 20D illustrate the T_(g=2,h=2) node 2025,T_(g=2,(h−1)=1) node 2024, and the T_(g=2,(h+1)=3) node 2026,respectively, as used in the load path 2000 of Reference to nodes of theWAM16L network 700 is also shown as the load network of the exemplaryconfiguration 1700 of FIG. 17 for clarity of discussion. FIG. 20Billustrates an exemplary memory T node system 2040 for the T_(g=2,h=2)node 755 in accordance with the present invention. The T node system2040 comprises expanded details of exemplary node T22 755 of FIG. 7, forexample, and memory nodes M12 731, M22 735, and M32 739, also of FIG. 7.The node T22 755 comprises a decoder 2041 having node operation (NodeOp)inputs 2042, three node function units 2044-2046 and a multiplexer 2053.The three node function units 2044-2046 comprises three groups of threetwo-input multipliers 2047-2049, three three-input adders 2050-2052, andthree multiplexers 2054-2056. The node T22 755 is coupled to the threememory nodes 731, 735, and 739 which supply the weights and a currentneuron value. As controlled by the NodeOp inputs 2042 and decoder 2041,the multipliers 2047-2049 are configured to multiply their input valuesand provide the results as input to the corresponding three-input adders2050-2052 that are configured to provide a sum of the weighted neuronnode results. The three-input adders 2050-2052 are coupled tocorresponding multiplexers 2054-2056. The multiplexer 2053 may beconfigured to select an output from one of the memories M12 731, M22735, and M32 739 which is applied as an input to multiplexers 2054-2056.Under control of the decoder 2041, the multiplexers 2054-2056 areconfigured to select an output of the three-input adders 2050-2052,respectively, or an output from the multiplexer 2053.

Current neuron values and weight values are stored in the memory nodesand may be formatted as 8-bit or 16-bit data values or for applicationspecific implementations may be specified as non-power of 2 data values,for example, to meet specific precision requirements in a fixed pointimplementation. Alternatively, the neuron and weight values may beformatted, for example, as single precision or double precision floatingpoint values. In one embodiment, a current neuron value and three weightvalues may be formatted as 8-bit data values and stored in a singleaddressable location in the memory nodes as 32-bits. Byte addressabilitymay also be supported for access to each individual value. In thisembodiment, the nine multipliers 2047-2049 may be 8×8 multipliers eachproducing, for example, a 16-bit result that is input to one of thethree three-input adders 2050-2052. For example, the three-input adder2051 generates, for example, a 16-bit summation of the three inputs,which may be a rounded or saturating fixed point result. In a differentembodiment, floating point arithmetic units may be used in a systemappropriately configured for floating point data types.

Operation of the 2D neural network based on the exemplary configuration1700 of FIG. 17 is described next for operation of neuron P22 whichoperates according to:P _(2,2) =F(W _((2,2),(1,1)) *P _(1,1) +W _((2,2),(2,1)) *P _(2,1) +W_((2,2),(3,1)) *P _(3,1) +W _((2,2),(1,2)) *P _(1,2) +W _((2,2),(2,2))*P _(2,2) +W _((2,2),(3,2)) *P _(3,2) +W _((2,2),(1,3)) *P _(1,3) +W_((2,2),(2,3)) *P _(2,3) +W _((2,2),(3,3)) *P _(3,3))The above equation for P_(2,2) can be viewed as a function F thatoperates on a summation of three parts. The portionW_((2,2),(1,2))*P_(1,2)+W_((2,2),(2,2))*P_(2,2)+W_((2,2),(3,2))*P_(3,2)is generated by node T22 755 of FIG. 20B. The portionW_((2,2),(1,1))*P_(1,1)+W_((2,2),(2,1))*P_(2,1)+W_((2,2),(3,1))*P_(3,1)is generated by node T21 754 of FIG. 20C. The portionW_((2,2),(1,3))*P_(1,3)+W_((2,2),(2,3))*P_(2,3)+W_((2,2),(3,3))*P_(3,3)is generated by node T23 756 of FIG. 20D.

In FIG. 20B, memory node M12 731 provides a current neuron value forP12, and weights W_((2,1),(1,2)), W_((2,2),(1,2)), and W_((2,3),(1,2)).Memory node M22 735 provides a current neuron value for P22 and weightsW_((2,1),(2,2)), W_((2,2),(2,2)), and W_((2,3),(2,2)). Memory node M32739 provides a current neuron value for P32 and weights W_((2,1),(3,2)),W_((2,2),(3,2)), and W_((2,3),(3,2)). The operation path for P22includes a multiplication W_((2,2),(1,2))*P_(1,2) which is generated inthe multiply group 2047, a multiplication W_((2,2),(2,2))*P_(2,2) whichis generated in the multiply group 2048, and another multiplicationW_((2,2),(3,2))*P_(3,2) which is generated in the multiply group 2049.The three multiplication results are added in the three input adder 2051to generateW_((2,2),(1,2))*P_(1,2)+W_((2,2),(2,2))*P_(2,2)+W_((2,2),(3,2))*P_(3,2)which is selected for output through multiplexer 2055 on T22B to L22output 2058.

FIG. 20C illustrates an exemplary memory T node system 2060 for theT_(2,1) node 754 in accordance with the present invention. In FIG. 20C,memory node M11 730 provides a current neuron value for P11, and weightsW_((2,0),(1,1)), W_((2,1),(1,1)), and W_((2,2),(1,1)). Memory node M21734 provides a current neuron value for P21 and weights W_((2,0),(2,1)),W_((2,1),(2,1)) and W_((2,2),(2,1)). Memory node M31 738 provides acurrent neuron value for P31 and weights W_((2,0),(3,1)),W_((2,1),(3,1)), and W_((2,2),(3,1)). The operation path for P22includes a multiplication W_((2,2),(1,1))*P_(1,1) which is generated inthe multiply group 2064, a multiplication W_((2,2),(2,1))*P_(2,1) whichis generated in the multiply group 2065, and another multiplicationW_((2,2),(3,1))*P_(3,1) which is generated in the multiply group 2066.The three multiplication results are added in the three input adder 2067to generateW_((2,2),(1,1))*P_(1,1)+W_((2,2),(2,1))*P_(2,1)+W_((2,2),(3,1))*P_(3,1)which is selected for output through multiplexer 2069 on T21C to L22output 2072.

FIG. 20D illustrates an exemplary memory T node system 2075 for theT_(2,3) node 756 in accordance with the present invention. In FIG. 20D,memory node M13 732 provides a current neuron value for P13, and weightsW_((2,2),(1,3)), W_((2,3),(1,3)), and W_((2,0),(1,3)). Memory node M23736 provides a current neuron value for P23 and weights W_((2,2),(2,3)),W_((2,3),(2,3)) and W_((2,0),(2,3)). Memory node M33 740 provides acurrent neuron value for P33 and weights W_((2,2),(3,3)),W_((2,3),(3,3)), and W_((2,0),(3,3)). The operation path for P22includes a multiplication W_((2,2),(1,3))*P_(1,3) which is generated inthe multiply group 2079, a multiplication W_((2,2),(2,3))*P_(2,3) whichis generated in the multiply group 2080, and another multiplicationW_((2,2),(3,3))*P_(3,3) which is generated in the multiply group 2081.The three multiplication results are added in the three input adder 2082to generateW_((2,2),(1,3))*P_(1,3)+W_((2,2),(2,3))*P_(2,3)+W_((2,2),(3,3))*P_(3,3)which is selected for output through multiplexer 2084 on T23A to L22output 2085.

FIG. 20E illustrates a node L22 2090 which provides a summation of the Tnode outputs generated in the previous stage in accordance with thepresent invention. The L22 node 2090 corresponds to the L22 node 775 ofFIG. 7. The node L22 2090 comprises a decoder 2091, node operation(NodeOp) inputs 2092, a three input adder 2093, and a multiplexer 2094.The T22B to L22 output 2058 from FIG. 20B, the T21C to L22 output 2072,and the T23A to L22 output 2085 are added in the three input adder 2093and selected for output through multiplexer 2094 on L22O to P22 output2095. Thus, the L22O to P22 output2095=W_((2,2),(1,1))*P_(1,1)+W_((2,2),(2,1))*P_(2,1)+W_((2,2),(3,1))*P_(3,1)+W_((2,2),(1,2))*P_(1,2)+W_((2,2),(2,2))*P_(2,2)+W_((2,2),(3,2))*P_(3,2)+W_((2,2),(1,3))*P_(1,3)+W_((2,2),(2,3))*P_(2,3)+W_((2,2),(3,3))*P_(3,3).The output of the L_(ghk) node 2008 of FIG. 20A provides a summation ofthe nine 3×3 adjacent weighted neuron values to the P_(ghk) node 2010.The neuron P22 receives the L22 node output and applies a sigmoidfunction F, for example, to generate a P22 neuron output.

In another example, the WAM64S network 900 of FIG. 9A is combined with aWAM64L network of similar construction to load networks as describedabove. The networks are configured with internal nodes having similarcapabilities as described for the combined network node 1650 withexpanded functions of FIG. 16B. FIG. 21A illustrates a load path 2100 toa neuron processor Pghk 2110 in accordance with the present invention.The load path 2100 is based on a 1 to N=3 adjacency of connections. TheL, T, and Z nodes comprise arithmetic functions similar to the functionsshown in FIGS. 20B-20E, described above. Each memory node supplies acurrent Pghk node value and weight values associated with the memorynode to Z nodes in the load network, first stage 2102. The load networkZ nodes multiply received weight values with received Pghk node valuesand provide a 3-to-one summation of the multiplied values which are sentto T nodes in a second stage 2104. In FIG. 21A, for example,Z_(g−1,h−1,k) node 2114 is configured to generate:Z _(g−1,h−1,k) =W _((g,h,k),(g−1,h−1,k−1)) *P _((g−1,h−1,k−1)) +W_((g,h,k),(g−1,h−1,k)) *P _((g−1,h−1,k)) +W _((g,h,k),(g−1,h−1,k+1)) *P_((g−1,h−1,k+1)),where P _(subscript) is the node value and the g, h, k values areassigned as above for wrap around connections.

Each T node is configured to receive Z node values from the coupled Znodes and to generate an N-to-one summation of the received Z nodevalues that is output from each T node and sent to L nodes, such as Lnode L_(g,h,k) 2108. For example, T_(g,h-1,k) node 2124 is configured togenerate, T_(g,h-1,k)=Z_(g-1,h-1,k)+Z_(g+1,h-1,k), which is a summationof the Z_(g-1,h-1,k) node 2114 output, Z_(g,h-1,k) node 2115 output, andZ_(g+1,h-,k) node 2116 output values. Each L node is configured toreceive T node values from the coupled T nodes and to generate anN-to-one summation of the received T node values that is output fromeach L node, such as L_(g,h,k) node 2108. The L_(g,h,k) 2108 isconfigured to generate, L_(g,h,k)=T_(g,h-1,l)+T_(g,h,k)+T_(g,h+1,k),which is a summation of the T_(g,h-1,k) node 2124 output, T_(g,h,k) node2125 output, and T_(g,h+1,k) node 2126 output values. The output of theL_(g,h,k) node 108 provides a summation of the twenty-seven 3×3×3adjacent weighted neuron values to the P_(g,h,k) node 2110.

Network nodes using a double adjacency 1 to 5 adjacency of connectionsmay be used for neural network computations. FIG. 21B illustrates anexemplary Z_(ghk) node 2140 for use in a 3 dimensional (3D) Wings neuralnetwork processor with each neuron having a 5×5×5 array of synapticweight values in accordance with the present invention. The Z_(ghk) node2140 comprises five node function units 2144-2148, a decoder 2141 havingnode operation (NodeOp) inputs 2142, and a multiplexer 2154. Each nodefunction unit, such as node function unit 2144, comprises a multipliergroup, such as multiplier group 2149 having five two-input multipliers,a five-input adder, such as five-input adder 2155, and an outputmultiplexer, such as multiplexer 2160. The Z_(ghk) node 2140 is coupledto five memory nodes M_(g,h,k−2), M_(g,h,k−1), M_(g,h,k), M_(g,h,k+1),M_(g,h,k+2). As controlled by NodeOp inputs 2142 and decoder 2141 thefive groups of multipliers are configured to multiply their input valuesand provide the results as input to the five-input adders to generate asum of the weighted neuron node outputs for the Z node which may beselected by the output multiplexer 2160 and output as ZghkA 2165.

For neural processing, instructions or NodeOp input signals are receivedat each of the M, Z, T, L, and P nodes to operate and control therespective nodes. In particular, the NodeOp signals, such as the NodeOpinputs 2142 may be instructions each having, for example the ALinstruction format 1020 of FIG. 10B, to specify an operation oroperations of a particular node, wherein such instruction orinstructions are decoded in decoder 2141. The Z nodes are coupled to Tnodes which provide a summation of the Z node outputs and the T nodesare coupled to L nodes which provide a summation of the T node outputs.The L nodes are coupled to neuron processor P nodes which generate aneuron output based on the summation of 5×5×5 (125) weighted neuroninputs for a neural processor constructed using the double adjacency 1to 5 adjacency of connections between nodes in each dimension. Otherlevels of adjacency and expanding into other dimensions of communicationare applicable for use in a Wings array system, including a Wings neuralnetwork (WNN) processor.

FIG. 22 illustrates a P_(g,h,k) node 2200 in accordance with the presentinvention. The P_(g,h,k) node 2200 comprises an array of function nodesF00node 2201-F22node 2209. Each of the function nodes, F00node2201-F22node 2209 is coupled to a corresponding storage node Fr00node2211-Fr22node 2219, which may be a buffer, a register, a register file,a queue, such as a first in first out (FIFO) memory, or a local memory,for example. The P_(g,h,k) node 2200 couples to a WAM store network, forexample, through the F00node 2201 and couples to a WAM load network alsothrough the function node F00node 2201, utilizing a buffer, a FIFOmemory, or the like.

Regarding function node F11node 2205, as an exemplary function noderepresenting the other function nodes in the array, a multiplexer 2255may be configured to select either an output of the function nodeF11node 2205 or select an output of the storage node Fr11node 2215. Theoutput of the multiplexer 2255 is gated by a three gate circuit 2245that provides three outputs coupled to first stage R nodes, R10 node2224, R11 node 2225, and R12 node 2226, which represents a couplingaccording to an adjacency in a first dimension. The function nodesF00node 2201-F22node 2209 and storage elements Fr00node 2211-Fr22node2219 are coupled to R nodes R00 node 2221-R22 node 2229, respectively,in a similar manner as described with the function node F11 node 2205.The R nodes R00node 2221-R22node 2229 are coupled to S nodes S00node2231-S22node 2239, according to an adjacency in a second dimension. TheS nodes S00 node 2231-S22 node 2239 are then coupled to the functionnodes F00node 2201-F22node 2209 and storage elements Fr00node2211-Fr22node 2219.

P_(g,h,k) node 2200 couples up to nine function nodes and up to ninestorage nodes in a 3×3 array using a 1 to 3 adjacency of connectionsnetwork. Each function node may include multiple execution units whichmay operate in parallel on fixed point or floating point data types. The3×3 array configuration allows chains of dependent instructions to beexecuted in pipeline fashion through the coupled nodes. For example, thesigmoid function may be applied to the input of function node F00node2201 received from the L_(ghk) node 2108 of a WAM load network togenerate a P_(ghk) neuron output. The sigmoid function may require achain of dependent instructions executing on each function node inpipeline fashion on a 3×3 array of function nodes. For example, a firstpart of the sigmoid function may be computed on F00node 2201 whichforwards results to one of the other function nodes and storage nodes inthe 3×3 array, such as function node F11node 2205 and storage nodeFr11node 2215 which computes a second part of the sigmoid function.While the second part is computed, a next sigmoid calculation may bestarted on the first node F00node 2201. The function node F11node 2205may then forward the second part results to another function node andstorage node, such as F10node 2204 and storage node Fr10 2214 whichcomputes a third part of the sigmoid function. While the third part iscomputed, the second part of the next sigmoid calculation may begin onthe function node F11node 2205. The sigmoid pipeline operations continuewith the final result forwarded in pipeline order to the WAM storenetwork.

FIG. 23A illustrates a hexagonal processor array 2300 organizedaccording to an INFORM coordinate system 2302 in accordance with thepresent invention. The INFORM coordinate system 2302 is based on axes at60 degree spacing resulting in six sectors 1-6. The IO axis 2304 of theINFORM coordinate system 2302 identifies an IO dimension ofcommunication, the NR axis 2306 identifies an NR dimension ofcommunication, and the FM axis 2308 identifies an FM dimension ofcommunication. The hexagonal processor array 2300 is laid out with rowpaths parallel to the FM dimension of communication, column pathsparallel to the IO dimension of communication, and diagonal pathsparallel to the NR dimension of communication. In FIG. 23A, nodes of thehexagonal processor array are placed at a (row, column) position in apositive 6^(th) sector 2310. It is noted that by placing a P node ateach of the following coordinates {(0,0), (0,1), (1,0), (3,4), (4,3),(4,4)} a 5×5 rhombus array would be obtained with each P node using theINFORM dimensions of communication. It is further noted that P nodes andtheir transpose P nodes are located in the NR dimension ofcommunication.

FIG. 23B illustrates a Wings hexagonal array memory (WHAM) storeconfiguration 2330 of the hexagonal processor array 2300 of FIG. 23Abased on a 1 to 3 adjacency of connections in each dimension ofcommunication with wrap around at the edge nodes of the hexagonal arrayin accordance with the present invention. The P nodes 2332 are coupledto R nodes 2334 according to the 1 to 3 adjacency of connections in theNR dimension of communication. The R nodes 2334 are coupled to the Snodes 2336 according to the 1 to 3 adjacency of connections in the FMdimension of communication. The S nodes 2336 are coupled to the V nodes2338 according to the 1 to 3adjacency of connections in the IO dimensionof communication. A communication path beginning from node P22, thecenter node in the hexagonal array 2300 of FIG. 23A to the memory nodes2340 is highlighted using outlined fonts. P22 is coupled to R nodes R31,R22, and R13 in the NR dimension of communication. R31 is coupled to Snodes S30, S31, and S32 in the FM dimension of communication. R22 iscoupled to S nodes S21, S22, and S23 in the FM dimension ofcommunication. R13 is coupled to S nodes S12, S13, and S14 in the FMdimension of communication. The S nodes S30, S31, S32, S21, S22, S23,S12, S13, and S14 are coupled to the V nodes 2338 in the IO dimension ofcommunication. Each V node is coupled to its corresponding memory Mnode. Other levels of adjacency and expanding into other dimensions ofcommunication are applicable for use in a Wings array system, includingthe Wings hexagonal array memory (WHAM) network.

FIG. 24 illustrates an exemplary WHAM19S network layout 2400 of thehexagonal processor array 2300 of FIG. 23A based on a 1 to 3 adjacencyof connections in each dimension of communication with wrap around atthe edge nodes of the hexagonal array in accordance with the presentinvention. The P nodes, M nodes, and network R, S, and V nodes arecoupled according to the dimensions of the INFORM coordinate system 2302of FIG. 23A. The P nodes are coupled to first stage R nodes acrossdiagonal paths parallel to the NR dimension of communication. Forexample, P nodes P40, P31, P22, P13, and P04 are coupled to R nodes R40,R31, R22, R13, and R04 as shown in the highlighted dashed box 2402. Thefirst stage R nodes are coupled to the second stage S nodes across rowpaths parallel to the FM dimension of communication. For example, Rnodes R20, R21, R22, R23, and R24 are coupled to S nodes S20, S21, S22,S23, and S24 as shown in the highlighted dashed box 2404. The secondstage S nodes are coupled to the third stage V nodes across column pathsparallel to the IO dimension of communication. For example, S nodes S02,S12, S22, S32, and S42 are coupled to V nodes V02, V12, V22, V32, andV42 as shown in the highlighted dashed box 2406. In an exemplaryimplementation, the P nodes and first stage R nodes may be organized onone layer of a multi-layer silicon chip. A different layer of the chipmay be utilized for the coupling between the first stage R nodes and thesecond stage S nodes. A different layer of the chip may be utilized forthe coupling between the second stage S nodes and the third stage Vnodes. The M nodes may be configured on the same layer with the thirdstage V nodes or on a different layer, such as the top layer of thechip. In such an organization the M nodes may be overlaid upon the Pnodes.

FIG. 25A illustrates a first exemplary Wings packet format 2500 inaccordance with the present invention. The first exemplary Wings packetformat 2500 comprises a 160bit (160b) eleven instruction packet 2502which comprises eight 12b arithmetic/logic (AL) instructions 2503, eachhaving, for example the AL instruction format 1020 of FIG. 10B, a 19bstore instruction 2504, having the store instruction format 1040 of FIG.10C, two 19b load instructions 2505 and 2506, each having the loadinstruction format 1060 of FIG. 10D, and with a 7b packet operation code2507. The 160b eleven instruction packet 2502 may be partitioned into amemory packet 2510 and a function packet 2512. The memory packet 2510comprises two load instructions each of which may be expanded to a dualload instruction format and a store instruction. The function packet2512 comprises 96 bits of AL type instructions which may comprise aplurality of different format function instructions as indicated in thefunction list 2514. For example, different formats of the functioninstructions in the function list 2514 may include the eight 12b ALinstructions, six 16b AL instructions, four 24b AL instructions, three32b AL instructions, or two 48b AL instructions. Other variations inpacket formats, memory packet formats and function packet formats may beused depending on application requirements. In the function packet 2512,an AL instruction may be paired with one or more load instructions, oneor more store instructions, and/or one or more of the other ALinstructions. For an AL instruction paired with a load instruction, asource operand, for example, may be provided by the coupled WAM loadnetwork to a function or storage node in a P node. For an AL instructionpaired with a store instruction, a data value, for example, may beprovided from a function or a storage node in a P node to the coupledWAM store network. For an AL instruction paired with one or more of theother AL instructions, a source operand, for example, may be provided bya function node associated with the paired AL instruction and a result,for example, may be provided to a function or storage node for use byanother paired AL instruction. AL instructions may be treated asbuilding block instructions that couple with load, store, and other ALinstructions. The instruction specific bits 1026 of FIG. 10B may be usedto designate a path for receiving a source operand and a path forcommunicating a result.

FIG. 25B illustrates a second exemplary Wings packet format 2530 inaccordance with the present invention. The second exemplary Wings packetformat 2530 comprises a 190bit (190b) fourteen instruction packet 2532which comprises eight 12b arithmetic/logic (AL) instructions 2533, eachhaving, for example the AL instruction format 1020 of FIG. 10B, sixmemory and network operate (M&N) instructions 2534, and an 8b packetoperation code 2535. The fourteen instruction packet 2532 may bepartitioned into a function packet 2537 and a M&N packet 2540. Thefunction packet 2537 comprises 96 bits of AL type instructions which maycomprise a plurality of different format function instructions asindicated in the function list 2538. The M&N packet 2540 comprises a 19bload instruction 2541, a network multiply (Mpy) AL type instruction2542, a first network add AL type instruction 2543, a second network addAL type instruction 2544, a 19b store instructions 2545, and a networkmultiplex (Mpx) AL type instruction 2546. The 19b load instruction 2541may have the load instruction format 1060 of FIG. 10D, the 19b storeinstruction 2545 may have the store instruction format 1040 of FIG. 10C,and each network AL type instruction 2542-2544 and 2546 may have the ALinstruction format 1020 of FIG. 10B. The 19b load instruction 2541 mayalso be a dual load type of instruction.

The 190 b fourteen instruction packet 2532 illustrates an exemplary setof instructions useful to operate and control nodes, such as, theexemplary Z_(ghk) node 2140 of FIG. 21B for use in a 3 dimensional (3D)Wings neural network processor with each neuron having a 5×5×5 array ofsynaptic weight values. The function packet 2537 may be dispatched tooperate and control neuron P nodes and the M&N packet 2540 may bedispatched to operate and control the memory and network nodes. Forexample, the 19 b load instruction 2541 may be dispatched to memorynodes configured to execute the received load instruction and providethe P and weight values to coupled Z_(ghk) nodes, such as the Z_(ghk)node 2140. The network multiply (Mpy) AL type instruction 2542 may bedispatched to each coupled Z_(ghk) node, such as the Z_(ghk) node 2140,configured to execute the received network Mpy instruction and provide asummation of weighted input values on each Z_(ghk) node output. Thefirst network add AL type instruction 2543 may be dispatched to eachcoupled T node and the second network add AL type instruction 2544 maybe dispatched to each coupled L node. Each of the coupled T nodes and Lnodes are configured to execute the instruction received and provide asummation of the 5×5×5 weighted neuron inputs. The neuron P nodes areconfigured to execute the instructions in the function packet 2537 togenerate a sigmoid type output, for example. The sigmoid type outputthen is coupled to a Wings store network using the double adjacency 1 to5 adjacency of connections to communicate the neuron values for storagein coupled memory nodes. Each of the Wings store network nodes isconfigured to execute the 12 b Mpx instruction 2546 to pass the sigmoidtype output to the memory nodes that are configured to execute the 19 bstore instruction 2545 and store the sigmoid type output in theappropriate specified location in preparation for another neural networkoperation. It is noted that these operations may be pipelined across the3D Wings neural network processor in stages according to, for examplethe instruction order specified in the 190 b fourteen instruction packet2532. It is noted that in comparison with instruction processors havinga 32-bit instruction set, such operations would require at leastfourteen 32-bit instructions, if not more, requiring storage for 14×32b=448-bits as compared to the 190-bits used in the exemplary neuralprocessor of the present invention.

FIG. 26 illustrates an exemplary WAM processor 2600 in accordance withthe present invention. The WAM processor 2600 comprises a memoryhierarchy 2602, a fetch and dispatch unit 2604, a plurality of threadcontrol units 2606, a plurality of load store packet units 2608, aplurality of ALU packet units 2610, and a processor memory array 2612,such as the processor memory layout 1700 of FIG. 17. The processormemory array 2612 is illustrated as an exemplary 4×4 organization thoughnot limited to such an organization and larger array multi-dimensionalorganizations may be utilized. For example, in a G×H×K organization,each of the P nodes may be configured, for example, as the 3×3 P_(g,h,k)node 2200 of FIG. 22. The thread control units 2606 may be configured tooperate as a single thread control for SIMD operation of the processormemory array 2612. The thread control units 2606 may alternatively beprogrammed to operate with multiple threads, such as four threads A-Dillustrated in FIG. 26. The memories in the processor memory array 2612are the shared Wings array memories accessible by the processors asdiscussed above.

While the present invention is disclosed in a presently preferredcontext, it will be recognized that the teachings of the presentinvention may be variously embodied consistent with the disclosure andclaims. By way of example, the present invention is applicable toregister based RISC type processors acting as the processor nodes thatcommunicate through a shared global memory. In another example, thenetwork 1206 of FIG. 12A may be implemented with various types ofnetworks while maintaining the split organization of the processor node1200 embodiment of the present invention. It will be recognized that thepresent teachings may be used for multi-dimensional data analysis andmay be adapted to other present and future architectures to which theymay be beneficial.

I claim:
 1. A network of nodes organized in stages according todimensions of a GxH matrix of nearest neighbor connected elements havingconnectivity within the stages according to a 1 to N adjacency ofconnections between elements in corresponding dimensions of the GxHmatrix which includes wrap around adjacent elements, the connectivitywithin the stages includes connections between nodes in the sameposition, N a positive odd integer, and G≧N and H≧N, the networkcomprising: groups of A_(g,h) nodes, each group having a different gthat is the same for each A_(g,h) node in that group, gε{0,1, . . . ,G−1} and for each group, hε{0,1, . . . , H−1}, and each A_(g,h) nodeoperable to output a data value; groups of R_(g,h) nodes, each grouphaving a different g that is the same for each R_(g,h) node in thatgroup, gε{0,1, . . . , G−1} and for each group, hε{0,1, . . . , H−1},each group of R_(g,h) nodes directly coupled to a corresponding group ofA_(g,h) nodes according to a 1 to N adjacency of nodes in a firstdimension, wherein each R_(g,h) node in a group of R_(g,h) nodes isoperable to select a data value solely from directly coupled nodesA_(g,h−└N/2┘), . . . , A_(g,h−2), A_(g,h−1), A_(g,h), A_(g,h+1),A_(g,h+2), . . . , A_(g,h+└N/2┘) and to output the R_(g,h) node selecteddata value, wherein for a selected value of N, the A_(g,h−└N/) _(2┘), .. . , A_(g,h−2,)A_(g,h−1) sequence of nodes has └N/2┘ nodes and for N>1the last node is A_(g,h−└N/2) and the A_(g,h+1), A_(g,h+2), . . . ,A_(g,h+└N/2┘) sequence of nodes has └N/2┘ nodes and for N>1 the lastnode is A_(g,h+└N/2┘); and groups of S_(g,h) nodes, each group having adifferent g that is the same for each S_(g,h) node in that group,gε{0,1, . . . ,G−1} and for each group, hε{0,1, . . . ,H−1 }, each groupof S_(g,h) nodes directly coupled to groups of R_(g,h) nodes accordingto a 1 to N adjacency of nodes in a second dimension, wherein eachS_(g,h) node is operable to select a data value solely from directlycoupled nodes R_(g−└N/2┘,h), . . . , R_(g−2,h), R_(g−1,h), R_(g,h),R_(g+1,h), R_(g+2,h), . . . , R_(g+└N/2┘,h) and to output the S_(g,h)node selected data value, wherein for the selected value of N, theR_(g−└N/2┘,h), . . . , R_(g−2,h), R_(g−1,h) sequence of nodes has└N/2┘nodes and for N>1 the last node is R_(g−└N/2┘,h), the R_(g+1,h),R_(g+2,h), . . . , R_(g+└N/2┘,h) sequence of nodes has └N/2┘ nodes andfor N>1 the last node is R_(g+└N/2┘,h), and └N/2┘ is the floor of N/2which is the largest integer less than N/2.
 2. The network of claim 1,further comprising: a plurality of B_(g,h) nodes, gε{0,1, . . . ,G−1}and hε{0,1, . . . ,H−1}, each B_(g,h) node directly coupled to acorresponding S_(g,h) node.
 3. The network of claim 2, wherein eachB_(g,h) node is overlaid upon a corresponding A_(g,h) node.
 4. Thenetwork of claim 2, wherein each B_(g,h) node is a processor that isoperable to select a path through a directly coupled R_(g,h) node tocommunicate the R_(g,h) node selected data value and to select a paththrough a directly coupled S_(g,h) node to communicate the S_(g,h) nodeselected data value in response to the B_(g,h) processor executing amemory access instruction.
 5. The network of claim 2, wherein eachB_(g,h) node is a storage node that is operable to select a path througha directly coupled R_(g,h) node to communicate the R_(g,h) node selecteddata value and to select a path through a directly coupled S_(g,h) nodeto communicate the S_(g,h) node selected data value in response to theB_(g,h) storage node executing a memory access instruction.
 6. Thenetwork of claim 1, wherein each A_(g,h) node is a processor that isoperable to select a path through a directly coupled R_(g,h) node tocommunicate the R_(g,h) node selected data value and to select a paththrough a directly coupled S_(g,h) node to communicate the S_(g,h) nodeselected data value in response to the A_(g,h) processor executing amemory access instruction.
 7. The network of claim 1, wherein eachA_(g,h) node is a storage node that is operable to select a path througha directly coupled R_(g,h) node to communicate the R_(g,h) node selecteddata value and to select a path through a directly coupled S_(g,h) nodeto communicate the S_(g,h) node selected data value in response to theA_(g,h) storage node executing a memory access instruction.
 8. Thenetwork of claim 1, wherein the R_(g,h) nodes and the S_(g,h) nodescomprise: a function unit selectively operable to execute a function onone or more data values received from the directly coupled nodes toproduce an output data value.
 9. The network of claim 1, furthercomprising: a plurality of processor (P_(g,h)) nodes, gε{0,1, . . .,G−1} and hε{0,1, . . . ,H−1}, each P_(g,h) node directly coupled to acorresponding S_(g,h) node, wherein the A_(g,h) nodes are configured asmemory nodes and the R_(g,h) nodes and S_(g,h) nodes are configured withfunction units to support artificial intelligence and neural processing.10. The network of claim 9, wherein the P_(g,h) nodes are configured tooperate as neuron processors in a neural network model, each neuronprocessor having an N×N array of synaptic weight values stored in thecoupled A_(g,h) memory nodes, wherein N is determined from the 1 to Nadjacency of connections used in the G×H matrix of nodes.
 11. A networkof nodes organized in stages according to dimensions of communication inan array having D dimensions of communication and nearest neighborconnected elements, the network of nodes having connectivity within thestages according to adjacency of connections between elements incorresponding dimensions of communication of the array, the networkcomprising: a plurality of A nodes, each A node identified according toits corresponding position in the array and operable to output an A datavalue; and a plurality of D stages, one stage for each dimension ofcommunication of the D-dimensional network, nodes in each stageidentified according to positions of corresponding elements in thearray, each A node directly coupled solely to N₁ first stage nodes thatare adjacent nodes in a first dimension of communication, the N₁ firststage nodes include a first stage node in the same position as thedirectly coupled A node, each of the N₁ first stage nodes directlycoupled solely to N₂ second stage nodes that are adjacent nodes in asecond dimension of communication, the N₂ second stage nodes include asecond stage node in the same position as the directly coupled firststage node, and continuing until each of N_(D −1) D-1 stage nodes aredirectly coupled solely to N_(D) D stage nodes that are adjacent nodesin a D^(th) dimension of communication, the N_(D) D stage nodes includea D stage node in the same position as the directly coupled D-1 stagenode, wherein each stage node is configured to operate on a data valuereceived from a directly coupled node in a previous stage that wasinitiated by the A data value output from each of the A nodes and tooutput a result value from each D stage node, wherein N₁, N₂, . . . ,N_(D) are each a positive odd integer and are each less than or equal tothe number of elements in each associated dimension of communication ofthe array.
 12. The network of claim 11, wherein each A node is coupledto N₁*N₂*. . . N_(D) D stage nodes.
 13. The network of claim 11, whereineach node comprises: a function unit operable to execute a function onone or more data values received from the directly coupled nodes,wherein the function is specified by an instruction.
 14. The network ofclaiml 11, wherein the instruction identifies a path to a source operandand a path for a result.
 15. The network of claim 11, wherein the arrayincludes wrap around adjacent elements.
 16. The network of claim 11,wherein each A node comprises: an array of storage nodes, each storagenode coupled to other storage nodes by a plurality of A node stagesaccording to a 1 to N_(A) adjacency in each A node stage, wherein thearray of storage nodes is operable to output the A data value, N_(A)Usselected according to requirements of the A node, and N_(A) is apositive odd integer.
 17. The network of claim 11, wherein each D stagenode is overlaid upon a corresponding A node.
 18. The network of claim11, wherein the array is a hexagonal array having D equal to threedimensions of communication.
 19. A network of nodes organized in stagesaccording to dimensions of communication in a multidimensional arrayhaving D dimensions of communication and nearest neighbor connectedelements, nodes in the network identified according to positions ofcorresponding elements in the multidimensional array, the networkcomprising: a plurality of memory (M) nodes, each M node identifiedaccording to its position in the multidimensional array and operable tooutput an M data value; a plurality first stage nodes, each M nodedirectly coupled solely to N₁ first stage nodes that are adjacent nodesin a first dimension of communication and include a first stage node inthe same position as the directly coupled M node; a plurality of secondstage nodes, each first stage node directly coupled solely to N₂ secondstage nodes that are adjacent nodes in a second dimension ofcommunication and include a second stage node in the same position asthe directly coupled first stage node; and continuing up to a pluralityof D^(th) stage nodes, each D−1 stage node directly coupled solely toN_(D) D^(th) stage nodes that are adjacent nodes in a D^(th) dimensionof communication and include a D stage node in the same position as thedirectly coupled D−1 stage node, wherein each first stage node isconfigured to selectively operate on the M data value output from eachdirectly coupled M node and to output a first stage node result value,each second stage node is configured to selectively operate on the firststage node result value from each directly coupled first stage node andto output a second stage node result value, and continuing up to each Dstage node is configured to selectively operate on a D−1 stage noderesult value from each directly coupled D−1 stage node and to output a Dstage node result value, wherein N₁, N₂, . . . , N_(D) are each apositive odd integer and are each less than or equal to the number ofelements in each associated dimension of communication of the array. 20.The network of claim 19, wherein the multidimensional array is amultidimensional hexagonal array.
 21. The network of claim 19, whereinN₁, N₂, . . . , N_(D) are determined from a 1 to N-level of adjacency ofnodes used in constructing each associated stage in the network.
 22. Thenetwork of claim 19, wherein each directly coupled first stage nodecomprises: a first multiplier configured to multiply a stored firstprocessor (P) node output value with a first weight value to produce aweighted first P node output value, wherein the stored first P nodeoutput value combined with the first weight value comprises a first Mdata value accessed from a directly coupled first M node; a secondmultiplier configured to multiply a stored second P node output valuewith a second weight value to produce a weighted second P node outputvalue, wherein the stored second P node output value combined with thesecond weight value comprises a second M data value accessed from adirectly coupled second M node; a third multiplier configured tomultiply a stored third P node output value with a third weight value toproduce a weighted third P node output value, wherein the stored third Pnode output value combined with the third weight value comprises a thirdM data value accessed from a directly coupled third M node; and an adderconfigured to add the weighted first P node output value with theweighted second P node output value with the weighted third P nodeoutput value to produce a first sum of weighted P node values.
 23. Thenetwork of claim 22, wherein each directly coupled second stage nodecontinuing up to each directly coupled D stage node comprises: an adderconfigured to add a result value from each directly coupled node,wherein the result value from each directly coupled D stage node is asummation of weighted P node values.
 24. The network of claim 19,further comprising: a multiply add function in each directly coupledfirst stage node configured to operate on M data values accessed from N₁M nodes; an add function in each directly coupled second stage nodeconfigured to operate on N₂ result values from the N₂ directly coupledfirst stage nodes; continuing up to each directly coupled D stage node,an add function in each directly coupled D stage node configured tooperate on N_(D) result values from the directly coupled N_(D) D−1 stagenodes; and a function unit in a plurality of processor (p) nodes, each Pnode directly coupled to each D stage node and configured to operate onan output of the directly coupled D stage node to produce a P noderesult, wherein each of the P node results is stored in a correspondingM node as part of the M data values.